Lib/lib-old/ni.py

   1 """New import scheme with package support.
   2
   3 Quick Reference
   4 ---------------
   5
   6 - To enable package support, execute "import ni" before importing any
   7   packages.  Importing this module automatically installs the relevant
   8   import hooks.
   9
  10 - To create a package named spam containing sub-modules ham, bacon and
  11   eggs, create a directory spam somewhere on Python's module search
  12   path (i.e. spam's parent directory must be one of the directories in
  13   sys.path or $PYTHONPATH); then create files ham.py, bacon.py and
  14   eggs.py inside spam.
  15
  16 - To import module ham from package spam and use function hamneggs()
  17   from that module, you can either do
  18
  19     import spam.ham             # *not* "import spam" !!!
  20     spam.ham.hamneggs()
  21
  22   or
  23
  24     from spam import ham
  25     ham.hamneggs()
  26
  27   or
  28
  29     from spam.ham import hamneggs
  30     hamneggs()
  31
  32 - Importing just "spam" does not do what you expect: it creates an
  33   empty package named spam if one does not already exist, but it does
  34   not import spam's submodules.  The only submodule that is guaranteed
  35   to be imported is spam.__init__, if it exists.  Note that
  36   spam.__init__ is a submodule of package spam.  It can reference to
  37   spam's namespace via the '__.' prefix, for instance
  38
  39     __.spam_inited = 1          # Set a package-level variable
  40
  41
  42
  43 Theory of Operation
  44 -------------------
  45
  46 A Package is a module that can contain other modules.  Packages can be
  47 nested.  Package introduce dotted names for modules, like P.Q.M, which
  48 could correspond to a file P/Q/M.py found somewhere on sys.path.  It
  49 is possible to import a package itself, though this makes little sense
  50 unless the package contains a module called __init__.
  51
  52 A package has two variables that control the namespace used for
  53 packages and modules, both initialized to sensible defaults the first
  54 time the package is referenced.
  55
  56 (1) A package's *module search path*, contained in the per-package
  57 variable __path__, defines a list of *directories* where submodules or
  58 subpackages of the package are searched.  It is initialized to the
  59 directory containing the package.  Setting this variable to None makes
  60 the module search path default to sys.path (this is not quite the same
  61 as setting it to sys.path, since the latter won't track later
  62 assignments to sys.path).
  63
  64 (2) A package's *import domain*, contained in the per-package variable
  65 __domain__, defines a list of *packages* that are searched (using
  66 their respective module search paths) to satisfy imports.  It is
  67 initialized to the list consisting of the package itself, its parent
  68 package, its parent's parent, and so on, ending with the root package
  69 (the nameless package containing all top-level packages and modules,
  70 whose module search path is None, implying sys.path).
  71
  72 The default domain implements a search algorithm called "expanding
  73 search".  An alternative search algorithm called "explicit search"
  74 fixes the import search path to contain only the root package,
  75 requiring the modules in the package to name all imported modules by
  76 their full name.  The convention of using '__' to refer to the current
  77 package (both as a per-module variable and in module names) can be
  78 used by packages using explicit search to refer to modules in the same
  79 package; this combination is known as "explicit-relative search".
  80
  81 The PackageImporter and PackageLoader classes together implement the
  82 following policies:
  83
  84 - There is a root package, whose name is ''.  It cannot be imported
  85   directly but may be referenced, e.g. by using '__' from a top-level
  86   module.
  87
  88 - In each module or package, the variable '__' contains a reference to
  89   the parent package; in the root package, '__' points to itself.
  90
  91 - In the name for imported modules (e.g. M in "import M" or "from M
  92   import ..."), a leading '__' refers to the current package (i.e.
  93   the package containing the current module); leading '__.__' and so
  94   on refer to the current package's parent, and so on.  The use of
  95   '__' elsewhere in the module name is not supported.
  96
  97 - Modules are searched using the "expanding search" algorithm by
  98   virtue of the default value for __domain__.
  99
 100 - If A.B.C is imported, A is searched using __domain__; then
 101   subpackage B is searched in A using its __path__, and so on.
 102
 103 - Built-in modules have priority: even if a file sys.py exists in a
 104   package, "import sys" imports the built-in sys module.
 105
 106 - The same holds for frozen modules, for better or for worse.
 107
 108 - Submodules and subpackages are not automatically loaded when their
 109   parent packages is loaded.
 110
 111 - The construct "from package import *" is illegal.  (It can still be
 112   used to import names from a module.)
 113
 114 - When "from package import module1, module2, ..." is used, those
 115     modules are explicitly loaded.
 116
 117 - When a package is loaded, if it has a submodule __init__, that
 118   module is loaded.  This is the place where required submodules can
 119   be loaded, the __path__ variable extended, etc.  The __init__ module
 120   is loaded even if the package was loaded only in order to create a
 121   stub for a sub-package: if "import P.Q.R" is the first reference to
 122   P, and P has a submodule __init__, P.__init__ is loaded before P.Q
 123   is even searched.
 124
 125 Caveats:
 126
 127 - It is possible to import a package that has no __init__ submodule;
 128   this is not particularly useful but there may be useful applications
 129   for it (e.g. to manipulate its search paths from the outside!).
 130
 131 - There are no special provisions for os.chdir().  If you plan to use
 132   os.chdir() before you have imported all your modules, it is better
 133   not to have relative pathnames in sys.path.  (This could actually be
 134   fixed by changing the implementation of path_join() in the hook to
 135   absolutize paths.)
 136
 137 - Packages and modules are introduced in sys.modules as soon as their
 138   loading is started.  When the loading is terminated by an exception,
 139   the sys.modules entries remain around.
 140
 141 - There are no special measures to support mutually recursive modules,
 142   but it will work under the same conditions where it works in the
 143   flat module space system.
 144
 145 - Sometimes dummy entries (whose value is None) are entered in
 146   sys.modules, to indicate that a particular module does not exist --
 147   this is done to speed up the expanding search algorithm when a
 148   module residing at a higher level is repeatedly imported (Python
 149   promises that importing a previously imported module is cheap!)
 150
 151 - Although dynamically loaded extensions are allowed inside packages,
 152   the current implementation (hardcoded in the interpreter) of their
 153   initialization may cause problems if an extension invokes the
 154   interpreter during its initialization.
 155
 156 - reload() may find another version of the module only if it occurs on
 157   the package search path.  Thus, it keeps the connection to the
 158   package to which the module belongs, but may find a different file.
 159
 160 XXX Need to have an explicit name for '', e.g. '__root__'.
 161
 162 """
 163
 164
 165 import imp
 166 import sys
 167 import __builtin__
 168
 169 import ihooks
 170 from ihooks import ModuleLoader, ModuleImporter
 171
 172
 173 class PackageLoader(ModuleLoader):
 174
 175     """A subclass of ModuleLoader with package support.
 176
 177     find_module_in_dir() will succeed if there's a subdirectory with
 178     the given name; load_module() will create a stub for a package and
 179     load its __init__ module if it exists.
 180
 181     """
 182
 183     def find_module_in_dir(self, name, dir):
 184         if dir is not None:
 185             dirname = self.hooks.path_join(dir, name)
 186             if self.hooks.path_isdir(dirname):
 187                 return None, dirname, ('', '', 'PACKAGE')
 188         return ModuleLoader.find_module_in_dir(self, name, dir)
 189
 190     def load_module(self, name, stuff):
 191         file, filename, info = stuff
 192         suff, mode, type = info
 193         if type == 'PACKAGE':
 194             return self.load_package(name, stuff)
 195         if sys.modules.has_key(name):
 196             m = sys.modules[name]
 197         else:
 198             sys.modules[name] = m = imp.new_module(name)
 199         self.set_parent(m)
 200         if type == imp.C_EXTENSION and '.' in name:
 201             return self.load_dynamic(name, stuff)
 202         else:
 203             return ModuleLoader.load_module(self, name, stuff)
 204
 205     def load_dynamic(self, name, stuff):
 206         file, filename, (suff, mode, type) = stuff
 207         # Hack around restriction in imp.load_dynamic()
 208         i = name.rfind('.')
 209         tail = name[i+1:]
 210         if sys.modules.has_key(tail):
 211             save = sys.modules[tail]
 212         else:
 213             save = None
 214         sys.modules[tail] = imp.new_module(name)
 215         try:
 216             m = imp.load_dynamic(tail, filename, file)
 217         finally:
 218             if save:
 219                 sys.modules[tail] = save
 220             else:
 221                 del sys.modules[tail]
 222         sys.modules[name] = m
 223         return m
 224
 225     def load_package(self, name, stuff):
 226         file, filename, info = stuff
 227         if sys.modules.has_key(name):
 228             package = sys.modules[name]
 229         else:
 230             sys.modules[name] = package = imp.new_module(name)
 231         package.__path__ = [filename]
 232         self.init_package(package)
 233         return package
 234
 235     def init_package(self, package):
 236         self.set_parent(package)
 237         self.set_domain(package)
 238         self.call_init_module(package)
 239
 240     def set_parent(self, m):
 241         name = m.__name__
 242         if '.' in name:
 243             name = name[:name.rfind('.')]
 244         else:
 245             name = ''
 246         m.__ = sys.modules[name]
 247
 248     def set_domain(self, package):
 249         name = package.__name__
 250         package.__domain__ = domain = [name]
 251         while '.' in name:
 252             name = name[:name.rfind('.')]
 253             domain.append(name)
 254         if name:
 255             domain.append('')
 256
 257     def call_init_module(self, package):
 258         stuff = self.find_module('__init__', package.__path__)
 259         if stuff:
 260             m = self.load_module(package.__name__ + '.__init__', stuff)
 261             package.__init__ = m
 262
 263
 264 class PackageImporter(ModuleImporter):
 265
 266     """Importer that understands packages and '__'."""
 267
 268     def __init__(self, loader = None, verbose = 0):
 269         ModuleImporter.__init__(self,
 270         loader or PackageLoader(None, verbose), verbose)
 271
 272     def import_module(self, name, globals={}, locals={}, fromlist=[]):
 273         if globals.has_key('__'):
 274             package = globals['__']
 275         else:
 276             # No calling context, assume in root package
 277             package = sys.modules['']
 278         if name[:3] in ('__.', '__'):
 279             p = package
 280             name = name[3:]
 281             while name[:3] in ('__.', '__'):
 282                 p = p.__
 283                 name = name[3:]
 284             if not name:
 285                 return self.finish(package, p, '', fromlist)
 286             if '.' in name:
 287                 i = name.find('.')
 288                 name, tail = name[:i], name[i:]
 289             else:
 290                 tail = ''
 291             mname = p.__name__ and p.__name__+'.'+name or name
 292             m = self.get1(mname)
 293             return self.finish(package, m, tail, fromlist)
 294         if '.' in name:
 295             i = name.find('.')
 296             name, tail = name[:i], name[i:]
 297         else:
 298             tail = ''
 299         for pname in package.__domain__:
 300             mname = pname and pname+'.'+name or name
 301             m = self.get0(mname)
 302             if m: break
 303         else:
 304             raise ImportError, "No such module %s" % name
 305         return self.finish(m, m, tail, fromlist)
 306
 307     def finish(self, module, m, tail, fromlist):
 308         # Got ....A; now get ....A.B.C.D
 309         yname = m.__name__
 310         if tail and sys.modules.has_key(yname + tail): # Fast path
 311             yname, tail = yname + tail, ''
 312             m = self.get1(yname)
 313         while tail:
 314             i = tail.find('.', 1)
 315             if i > 0:
 316                 head, tail = tail[:i], tail[i:]
 317             else:
 318                 head, tail = tail, ''
 319             yname = yname + head
 320             m = self.get1(yname)
 321
 322         # Got ....A.B.C.D; now finalize things depending on fromlist
 323         if not fromlist:
 324             return module
 325         if '__' in fromlist:
 326             raise ImportError, "Can't import __ from anywhere"
 327         if not hasattr(m, '__path__'): return m
 328         if '*' in fromlist:
 329             raise ImportError, "Can't import * from a package"
 330         for f in fromlist:
 331             if hasattr(m, f): continue
 332             fname = yname + '.' + f
 333             self.get1(fname)
 334         return m
 335
 336     def get1(self, name):
 337         m = self.get(name)
 338         if not m:
 339             raise ImportError, "No module named %s" % name
 340         return m
 341
 342     def get0(self, name):
 343         m = self.get(name)
 344         if not m:
 345             sys.modules[name] = None
 346         return m
 347
 348     def get(self, name):
 349         # Internal routine to get or load a module when its parent exists
 350         if sys.modules.has_key(name):
 351             return sys.modules[name]
 352         if '.' in name:
 353             i = name.rfind('.')
 354             head, tail = name[:i], name[i+1:]
 355         else:
 356             head, tail = '', name
 357         path = sys.modules[head].__path__
 358         stuff = self.loader.find_module(tail, path)
 359         if not stuff:
 360             return None
 361         sys.modules[name] = m = self.loader.load_module(name, stuff)
 362         if head:
 363             setattr(sys.modules[head], tail, m)
 364         return m
 365
 366     def reload(self, module):
 367         name = module.__name__
 368         if '.' in name:
 369             i = name.rfind('.')
 370             head, tail = name[:i], name[i+1:]
 371             path = sys.modules[head].__path__
 372         else:
 373             tail = name
 374             path = sys.modules[''].__path__
 375         stuff = self.loader.find_module(tail, path)
 376         if not stuff:
 377             raise ImportError, "No module named %s" % name
 378         return self.loader.load_module(name, stuff)
 379
 380     def unload(self, module):
 381         if hasattr(module, '__path__'):
 382             raise ImportError, "don't know how to unload packages yet"
 383         PackageImporter.unload(self, module)
 384
 385     def install(self):
 386         if not sys.modules.has_key(''):
 387             sys.modules[''] = package = imp.new_module('')
 388             package.__path__ = None
 389             self.loader.init_package(package)
 390             for m in sys.modules.values():
 391                 if not m: continue
 392                 if not hasattr(m, '__'):
 393                     self.loader.set_parent(m)
 394         ModuleImporter.install(self)
 395
 396
 397 def install(v = 0):
 398     ihooks.install(PackageImporter(None, v))
 399
 400 def uninstall():
 401     ihooks.uninstall()
 402
 403 def ni(v = 0):
 404     install(v)
 405
 406 def no():
 407     uninstall()
 408
 409 def test():
 410     import pdb
 411     try:
 412         testproper()
 413     except:
 414         sys.last_type, sys.last_value, sys.last_traceback = sys.exc_info()
 415         print
 416         print sys.last_type, ':', sys.last_value
 417         print
 418         pdb.pm()
 419
 420 def testproper():
 421     install(1)
 422     try:
 423         import mactest
 424         print dir(mactest)
 425         raw_input('OK?')
 426     finally:
 427         uninstall()
 428
 429
 430 if __name__ == '__main__':
 431     test()
 432 else:
 433     install()