azure 获取错误:[Errno 95]在数据块中写入zip文件时不支持操作

4urapxun  于 2023-03-31  发布在  其他
关注(0)|答案(1)|浏览(160)

在这里,我试图压缩文件,并写入一个文件夹(挂载点)使用下面的代码在Databricks。

# List all files which need to be compressed
import os
modelPath  = '/dbfs/mnt/temp/zip/'
filenames = [os.path.join(root, name) for root, dirs, files in os.walk(top=modelPath , topdown=False) for name in files]
print(filenames)

zipPath = '/dbfs/mnt/temp/compressed/demo.zip'
import zipfile
with zipfile.ZipFile(zipPath, 'w') as myzip:
  for filename in filenames:
    print(filename)
    print(myzip)
    myzip.write(filename)

但我收到错误,因为[Errno 95]不支持操作。
错误详细信息

OSError                                   Traceback (most recent call last)
<command-2086761864237851> in <module>
     15     print(myzip)
---> 16     myzip.write(filename)

/usr/lib/python3.8/zipfile.py in write(self, filename, arcname, compress_type, compresslevel)
   1775             with open(filename, "rb") as src, self.open(zinfo, 'w') as dest:
-> 1776                 shutil.copyfileobj(src, dest, 1024*8)
   1777 

/usr/lib/python3.8/zipfile.py in close(self)
   1181                 self._fileobj.write(self._zinfo.FileHeader(self._zip64))
-> 1182                 self._fileobj.seek(self._zipfile.start_dir)
   1183 

OSError: [Errno 95] Operation not supported

During handling of the above exception, another exception occurred:

OSError                                   Traceback (most recent call last)
/usr/lib/python3.8/zipfile.py in close(self)
   1837                     if self._seekable:
-> 1838                         self.fp.seek(self.start_dir)
   1839                     self._write_end_record()

OSError: [Errno 95] Operation not supported

During handling of the above exception, another exception occurred:

OSError                                   Traceback (most recent call last)
OSError: [Errno 95] Operation not supported

During handling of the above exception, another exception occurred:

OSError                                   Traceback (most recent call last)
<command-2086761864237851> in <module>
     14     print(filename)
     15     print(myzip)
---> 16     myzip.write(filename)

/usr/lib/python3.8/zipfile.py in __exit__(self, type, value, traceback)
   1310 
   1311     def __exit__(self, type, value, traceback):
-> 1312         self.close()
   1313 
   1314     def __repr__(self):

/usr/lib/python3.8/zipfile.py in close(self)
   1841             fp = self.fp
   1842             self.fp = None
-> 1843             self._fpclose(fp)
   1844 
   1845     def _write_end_record(self):

/usr/lib/python3.8/zipfile.py in _fpclose(self, fp)
   1951         self._fileRefCnt -= 1
   1952         if not self._fileRefCnt and not self._filePassed:
-> 1953             fp.close()
   1954 
   1955

谁能帮我解决这个问题。
注意:这里我可以使用shutil压缩文件,但我想避免驱动程序,所以使用上述方法。

3pvhb19x

3pvhb19x1#

你没有提供你的挂载的细节,可能是Blob存储或ADLSv2,显然它不允许文件搜索。
看看这个简单的片段:

%python

path = '/dbfs/mnt/temp/testfile'

with open(path, "w") as f:
    f.write("test")
    f.seek(1)
    f.write("x")

with open(path, "r") as f:
    print(f.read())

它将在f.seek(1)处抛出“Operation not supported”。
path = '/tmp/testfile'重复同样的操作,您将得到正确的结果(“txst”)。
奇怪的是,www.example.com中的seekzipfile.py根本不应该被访问,看起来self._seekable返回了错误的值,我不确定这是库还是Azure的问题。
无论如何,只需在本地目录中创建归档文件,然后将其移动到挂载。

tempPath = '/tmp/demo.zip'
zipPath = '/dbfs/mnt/temp/compressed/demo.zip'
import zipfile
import os

with zipfile.ZipFile(tempPath, 'w') as myzip:
  for filename in filenames:
    print(filename)
    print(myzip)
    myzip.write(filename)

os.rename(tempPath, zipPath)

相关问题