← Back to team overview

pyexiv2-developers team mailing list archive

Re: memory problem

 

On 05/11/2010 08:20 AM, Olivier Tilloy wrote:

 Hello,

 Your description of the issue suggests that it may very well be an issue
 in pyexiv2 itself.
 It would help a lot the investigation if you could let us know what
 version of pyexiv2 you are using, and provide your script or a minimal
 script that reproduces this behaviour.

 Note that pyexiv2 0.2.0 had a nasty memory leak that was fixed in
 pyexiv2 0.2.1, could that be the same issue?

 Cheers,

 Olivier


 On 2010-05-11, S. J.<iform@xxxxxxxx>   wrote:

 Hi,

 I have written a small python app that uses the pyexiv2 library.
 However, when running the app, I see an interesting behavior.

 My app walks through a directory and its subfolders, renaming each file
 according to it exif data metadata.  It also writes the old filename to
 the xmp metadata fields as keywords.  The at first the app functions
 normally, memory usage is under control, the processor barely being used
 and near constant file I/O.  However, it came to a point where memory
 usage climbs, processor is maxed at 50% (only using one core). and disk
 I/O is very sporadic.

 I get error messages to the console such as the below....

 Warning: Directory NikonPreview, entry 0x0201: Data area exceeds data
 buffer, ignoring it.


 I am wondering what might be causing this.  I cannot reproduce the error
 or the conditions using the exiv2 binary for windows.  It seems to read
 all the metadata without errors.  So I am not sure whether this is a bug
 in exiv2 or pyexiv2.

 -Please Help
 iform

Hi,

After running a few more tests...it looks like the culprit is not
reading the metadata, but writing it.

Included is a sample script that has the writeTags function of my app.

Behavior is observed using pyexiv2 .21

Hope this helps
iform



import subprocess
import os,datetime
import pyexiv2
import gapclasses

def callExiv2(file):
    rcode = subprocess.call(["exiv2", "-pa", file])
    print "exiv2: %s" % rcode

def writeTags(filePath, relativePath, artistString=""):
    """Write EXIF artist tag to copied file"""
    print "\n\nEntered gapclasses.writeTags"
    keywords = []
    newT = pyexiv2.ImageMetadata(os.path.abspath(filePath))

    try:
        print "Reading data first try %s"%filePath
        newT.read()
        newT['Exif.Image.Artist'].value
        print "Read Success: %s"%filePath
    except KeyError:
        #Program will NOT remove existing values
        print "Setting artist string"
        newT['Exif.Image.Artist'] = artistString
        print "Set artist string"
        print "Generating Keyword List"
        keywords = os.path.normpath(relativePath).split(os.sep)
        print "Keyword list generated"
        try:
            print "Getting existing keywords"
            existingKeywords = newT['Xmp.dc.subject'].value
            print "Got keywords...now appending keyword list"
            newT['Xmp.dc.subject'] = existingKeywords + keywords
        except KeyError:
            print "KeyError...no existing keywords. Setting keywords"
            newT['Xmp.dc.subject'] = keywords
    except IOError:
        print "Unsupported file Type"


    print "FYI: Starting to attempt writing %s"%filePath
    try:
        print "Attempting to write all the data"
        print newT.write()
        print "Wrote data SUCCESSFUL %s"%filePath
    except:
        print "Write Failed!\n"

topdir = os.path.abspath("/home/ryan/testRAW")

for path, dirs, file in os.walk(topdir, topdown=False):
    for f in file:
        #callExiv2(os.path.join(path,f))
        fileName = os.path.join(path,f)
        writeTags(fileName, os.path.relpath(fileName), "Ryan Grout")


Follow ups