image loading performance problems with python and gobject -

i have script gtk(gobject) interface use posting photo blog.

i'm trying improve it's responsiveness loading images in background thread.

i've had no luck trying populate gdkpixbuf objects background thread, i've tried jams solid.

so alternate thought i'd read files in background thread , push them gdkpixbuf's on demand. approach has yielded surprising , rather depressing performance results make me wonder if i'm doing grossly wrong.

i'm playing lightly compressed jpegs off camera, tend around 3.8mb.

here's original blocking image load:

pb = gdkpixbuf.pixbuf.new_from_file(image_file) 

this averages 550ms, not huge, rather tedious if want flick through dozen images.

then split up, here's file read:

data = bytearray(open(self.image_file).read()) 

this averages 15ms, that's nice, kinda worrying, if can read file in 15ms other 535ms being spent on?

incidentally bytearray call exists because pixbufloader wouldn't accept data otherwise.

and pixbuf load:

pbl = gdkpixbuf.pixbufloader() pbl.write(data, len(data)) pbl.close() pb = pbl.get_pixbuf() 

this averages around 1400ms, 3 times longer letting gtk all.

am doing wrong here?

my guess: doing wrong. i've compared libjpeg-turbo gdk.pixbufloader , found virtually no speed differences. code used below.

for libjpeg-turbo (jpegload.c):

#include <assert.h> #include <stdio.h> #include <stdlib.h> #include <sys/time.h>  #include <jpeglib.h>  void decompress(file* fd) {   jsamparray buffer;   int row_stride;   struct jpeg_decompress_struct cinfo;   struct jpeg_error_mgr jerr;   cinfo.err = jpeg_std_error(&jerr);   jpeg_create_decompress(&cinfo);   jpeg_stdio_src(&cinfo, fd);   jpeg_read_header(&cinfo, true);   jpeg_start_decompress(&cinfo);   row_stride = cinfo.output_width * cinfo.output_components;   buffer = (*cinfo.mem->alloc_sarray)                 ((j_common_ptr) &cinfo, jpool_image, row_stride, 1);   while (cinfo.output_scanline < cinfo.output_height) {     (void) jpeg_read_scanlines(&cinfo, buffer, 1);   }   jpeg_finish_decompress(&cinfo);   jpeg_destroy_decompress(&cinfo); }  int main(int argc, char** argv) {   long len;   file *fd;   unsigned char *buf;   struct timeval start, end;   int i;   const int n = 100;   int delta;    /* read file cache in memory */   assert(argc == 2);   fd = fopen(argv[1], "rb");   fseek(fd, 0, seek_end);   len = ftell(fd);   rewind(fd);   buf = malloc(len);   assert(buf != null);   assert(fread(buf, 1, len, fd) == len);    gettimeofday(&start, null);   for(i = 0; < n; i++) {     rewind(fd);     decompress(fd);   }   gettimeofday(&end, null);   if(end.tv_sec > start.tv_sec) {     delta = (end.tv_sec - start.tv_sec - 1) * 1000;     end.tv_usec += 1000000;   }   delta += (end.tv_usec - start.tv_usec) / 1000;   printf("time spent in decompression: %d msec\n",          delta/n); } 

for python gdk (gdk_load.py):

import sys import gtk import time  def decompress(data):     pbl = gtk.gdk.pixbufloader()     pbl.write(data, len(data))     pbl.close()     return pbl.get_pixbuf()  data = open(sys.argv[1]).read()  n = 100 start = time.time() in xrange(n):     decompress(data) end = time.time() print "time spent in decompression: %d msec" % int((end - start) * 1000 / n) 

test run results:

$ gcc jpegload.c -ljpeg $ ./a.out dsc_8450.jpg  time spent in decompression: 75 msec $ python gdk_load.py dsc_8450.jpg  time spent in decompression: 75 msec $ identify dsc_8450.jpg  dsc_8450.jpg jpeg 3008x2000 3008x2000+0+0 8-bit directclass 2.626mb 0.000u 0:00.019 

edit: , test, using gi.repostiroy time:

import sys import time gi.repository import gdkpixbuf  def decompress(filename):     pb = gdkpixbuf.pixbuf.new_from_file(filename)     return pb  n = 100 start = time.time() in xrange(n):     decompress(sys.argv[1]) end = time.time() print "time spent in decompression: %d msec" % int((end - start) * 1000 / n) 

and results:

$ python gi_load.py dsc_8450.jpg  time spent in decompression: 74 msec 

gdkpixbuf.pixbufloader using gi.repository much slower "pure" gtk.gdk. code:

import sys import time gi.repository import gdkpixbuf  def decompress(data):     pbl = gdkpixbuf.pixbufloader()     pbl.write(data, len(data))     pbl.close()     return pbl.get_pixbuf()  data = bytearray(open(sys.argv[1]).read())  n = 100 start = time.time() in xrange(n):     decompress(data) end = time.time() print "time spent in decompression: %d msec" % int((end - start) * 1000 / n) 

results:

$ python gi_load.py dsc_8450.jpg  time spent in decompression: 412 msec 

but gdkpixbuf.pixbuf.new_from_file works fast pure c version using gi.repository, still either doing wrong, or expecting much.