python - Are urllib2 and httplib thread safe? -


i'm looking information on thread safety of urllib2 , httplib. official documentation (http://docs.python.org/library/urllib2.html , http://docs.python.org/library/httplib.html) lacks information on subject; word thread not mentioned there...

update

ok, not thread-safe out of box. what's required make them thread-safe or there scenario in can thread-safe? i'm asking because it's seems that

  • using separate openerdirector in each thread
  • not sharing http connection among threads

would suffice safely use these libs in threads. similar usage scenario proposed in question urllib2 , cookielib thread safety

httplib , urllib2 not thread-safe.

urllib2 not provide serialized access global (shared) openerdirector object, used urllib2.urlopen().

similarly, httplib not provide serialized access httpconnection objects (i.e. using thread-safe connection pool), sharing httpconnection objects between threads not safe.

i suggest using httplib2 or urllib3 alternative if thread-safety required.

generally, if module's documentation not mention thread-safety, assume not thread-safe. can @ module's source code verification.

when browsing source code determine whether module thread-safe, can start looking uses of thread synchronization primitives threading or multiprocessing modules, or use of queue.queue.

update

here relevant source code snippet urllib2.py (python 2.7.2):

_opener = none def urlopen(url, data=none, timeout=socket._global_default_timeout):     global _opener     if _opener none:         _opener = build_opener()     return _opener.open(url, data, timeout)  def install_opener(opener):     global _opener     _opener = opener 

there obvious race condition when concurrent threads call install_opener() , urlopen().

also, note calling urlopen() request object url parameter may mutate request object (see source openerdirector.open()), not safe concurrently call urlopen() shared request object.

all told, urlopen() thread-safe if following conditions met:

  • install_opener() not called thread.
  • a non-shared request object, or string used url parameter.

Comments

Popular posts from this blog

objective c - Change font of selected text in UITextView -

php - Accessing POST data in Facebook cavas app -

c# - Getting control value when switching a view as part of a multiview -