DMV/CCM – todo-list / progress

2 TODO Adjacency and combining it with inner()

- -

Each DMV_Rule now has both a probN and a probA, for -adjacencies. inner() needs the correct one in each case. -

-Adjacency gives a problem with duplicate words/tags, eg. in the -sentence "a a b". If this has the dependency structure b->a₀->a₁, -then b is non-adjacent to a₀ and should use probN (for the LRStop and -the attachment of a₀), while the other rules should all use -probA. But within the e(0,2,b) we can't just say "oh, a has index 0 -so it's not adjacent to 2", since there's also an a at index 1, and -there's also a dependency structure b->a₁->a₀ for that. We want -both. And in possibly much more complex versions. -

-Ideas: -

-I first thought of decorating the individual words/tags in a -sentence with their indices, and perhaps just duplicating the -relevant rules (one for each index of the duplicate tags). But this -gives an explosion in attachment rules (although a contained -explosion, within the rules used in a sentence; but most sentences -will have at least two NN's so it will be a problem). -
-Then, I had a brilliant idea. Just let e(), the helper function of -inner(), parametrize for an extra pair of boolean values for whether -or not we've attached anything to the left or right yet ("yet" -meaning "below"). So now, e() has a chart of the form [s, t, LHS, -Lattach, Rattach], and of course e(s,t,LHS) is the sum of the four -possible values for (Lattach,Rattach). This makes e() lots more -complex and DMV-specific though, so it's been rewritten in -inner_dmv() in dmv.py. -
-But, the problem is that we still need to be able get inner -probabilites for only trees where the h at location loc_h is the -root of the dependency. The previous idea does not give that. So I -made another version of inner() that parametrizes for loc_h in the -chart (instead of Lattach and Rattach). loc_h also lets us check -whether we need probN or probA, in a much simpler fashion than the -Lattach / Rattach version. -
TODO document this adjacency stuff better
-
TODO [#A] test and debug my brilliant idea
-
- Works on simple 'h h' sentence, but not with P_STOP, why?
  -
-
DONE implement my brilliant idea.
-CLOSED: 2008-06-01 Sun 17:19
-e(sti) in dmv.py - -
DONE [#A] test inner() on sentences with duplicate words
-Works with eg. the sentence "h h h"

3 TODO [#A] P_STOP for IO/EM

2 TODO [#A] P_STOP and P_CHOOSE for IO/EM

dmv-P_STOP Remember: The P_STOP formula is upside-down (left-to-right also). @@ -144,49 +75,69 @@ b[(RBAR, n_h), 'h'] = h_.probA # h_ is RBAR stop rule b[(LRBAR, n_h), 'h'] = h_.probA * _ h_.probA -

How is the P_STOP formula different given other values for dir and adj?
+

P_STOP formulas for various dir and adj:
Assuming this:

P_STOP(STOP|h,L,non_adj) = ∑_corpus ∑_s<loc(h) ∑_t -inner(s,t,(LRBAR,h)…) / ∑_corpus ∑_s<loc(h) ∑_t inner(s,t,(RBAR,h)…) +inner(s,t,_h_, …) / ∑_corpus ∑_s<loc(h) ∑_t +inner(s,t,h_, …)
P_STOP(STOP|h,L,adj) = ∑_corpus ∑_s=loc(h) ∑_t -inner(s,t,(LRBAR,h)…) / ∑_corpus ∑_s=loc(h) ∑_t inner(s,t,(RBAR,h)…) +inner(s,t,_h_, …) / ∑_corpus ∑_s=loc(h) ∑_t +inner(s,t,h_, …)
P_STOP(STOP|h,R,non_adj) = ∑_corpus ∑_s ∑_t>loc(h) -inner(s,t,(LRBAR,h)…) / ∑_corpus ∑_s ∑_t>loc(h) inner(s,t,(RBAR,h)…) +inner(s,t,_h_, …) / ∑_corpus ∑_s ∑_t>loc(h) +inner(s,t,h_, …)
P_STOP(STOP|h,R,adj) = ∑_corpus ∑_s ∑_t=loc(h) -inner(s,t,(LRBAR,h)…) / ∑_corpus ∑_s ∑_t=loc(h) inner(s,t,(RBAR,h)…) - - +inner(s,t,_h_, …) / ∑_corpus ∑_s ∑_t=loc(h) +inner(s,t,h_, …)

(And P_STOP(-STOP|…) = 1 - P_STOP(STOP|…) )

P_CHOOSE formula:
+Assuming this: +

+P_CHOOSE(a|h,L) = ∑_corpus ∑_s<loc(h) ∑_t=loc(h) ∑_r<t +inner(s,r,_a_, …) / ∑_corpus ∑_s<loc(h) ∑_t=loc(h) +inner(s,t,h_,…) +
- +or t >= loc(h)? +
-

+ +

+P_CHOOSE(a|h,R) = ∑_corpus ∑_s=loc(h) ∑_r>s +inner(s,r,_a_,…) / ∑_corpus ∑_s=loc(h) ∑_t +inner(s,t,h, …) +

+or s => loc(h)? -

4 TODO P_CHOOSE for IO/EM

- -

Write the formulas! should be easy? -

+ + + + + + +

5 Initialization

3 Initialization

dmv-inits

@@ -194,19 +145,19 @@ inner(s,t,(LRBAR,h)…) / ∑_corpus ∑_s ∑

TODO Separate initialization to another file? PRETTIER
+
TODO Separate initialization to another file? PRETTIER
(It's rather messy.)
TOGROK CCM Initialization
+
TOGROK CCM Initialization
P_SPLIT used here… how, again?
DONE DMV Initialization probabilities
+
DONE DMV Initialization probabilities
(from initialization frequency)
DONE DMV Initialization frequencies
+
DONE DMV Initialization frequencies
CLOSED: 2008-05-27 Tue 20:04
- P_STOP
  +
- P_STOP
  P_STOP is not well defined by K&M. One possible interpretation given the sentence [det nn vb nn] is
```
@@ -223,7 +174,7 @@ the sentence [det nn vb nn] is
  f_{STOP}(-STOP|nn, L, non_adj) +0
 
```
  - TODO tweak
    +
  - TODO tweak
```
             f[head,  'STOP', 'LN'] += (i_h <= 1)     # first two words
             f[head, '-STOP', 'LN'] += (not i_h <= 1)     
@@ -271,7 +222,7 @@ for all heads:
 
```
- P_CHOOSE
  +
- P_CHOOSE
  Go through the corpus, counting distances between heads and arguments. In [det nn vb nn], we give

6 [#C] Deferred

4 [#C] Deferred

TODO if loc_h == t, no need to try right-attachment rules OPTIMIZE
+
TODO if loc_h == t, no need to try right-attachment rules OPTIMIZE
** TODO if loc_h == s, no need to try left-attachment rules :OPTIMIZE:
DONE inner_dmv() should disregard rules with heads not in sent OPTIMIZE
+
DONE inner_dmv() should disregard rules with heads not in sent OPTIMIZE
CLOSED: 2008-06-08 Sun 10:18
If the sentence is "nn vbd det nn", we should not even look at rules where @@ -334,9 +285,9 @@ attachment rules where both daughters are members of ['vbd','det'] same tag if there are no duplicate tags in the span, etc., that would be a lot of trouble for little potential gain).
TODO when reestimating P_STOP etc, remove rules with p < epsilon OPTIMIZE
+
TODO when reestimating P_STOP etc, remove rules with p < epsilon OPTIMIZE
TODO inner_dmv, short ranges and impossible attachment OPTIMIZE
+
TODO inner_dmv, short ranges and impossible attachment OPTIMIZE
If s-t <= 2, there can be only one attachment below, so don't recurse with both Lattach=True and Rattach=True. @@ -348,7 +299,7 @@ Lattach=False, Rattach=False. Put this in the loop under rewrite rules (could also do it in the STOP section, but that would only have an effect on very short sentences).
TODO clean up the module files PRETTIER
+
TODO clean up the module files PRETTIER
Is there better way to divide dmv and harmonic? There's a two-way dependency between the modules. Guess there could be a third file that imports both the initialization and the actual EM stuff, while a file @@ -359,7 +310,7 @@ containing constants and classes could be imported by all others:
TOGROK Some (tagged) sentences are bound to come twice OPTIMIZE
+
TOGROK Some (tagged) sentences are bound to come twice OPTIMIZE
Eg, first sort and count, so that the corpus [['nn','vbd','det','nn'], ['vbd','nn','det','nn'], @@ -374,7 +325,7 @@ frequency correctly. Is there much to gain here?
TOGROK tags as numbers or tags as strings? OPTIMIZE
+
TOGROK tags as numbers or tags as strings? OPTIMIZE
Need to clean up the representation.
@@ -387,9 +338,53 @@ initialization..

7 Expectation Maximation in IO/DMV-terms

5 Adjacency and combining it with inner()

+ +

Each DMV_Rule now has both a probN and a probA, for +adjacencies. inner() needs the correct one in each case. +

+Adjacency gives a problem with duplicate words/tags, eg. in the +sentence "a a b". If this has the dependency structure b->a₀->a₁, +then b is non-adjacent to a₀ and should use probN (for the LRStop and +the attachment of a₀), while the other rules should all use +probA. But within the e(0,2,b) we can't just say "oh, a has index 0 +so it's not adjacent to 2", since there's also an a at index 1, and +there's also a dependency structure b->a₁->a₀ for that. We want +both. And in possibly much more complex versions. +

+So now we call inner() for each location of a head, and on each +terminal, loc_h must equal s (and t). In the recursive attachment +calls, we use the locations (sentence indices) of words to the left or +right of the head in calls to inner(). loc_h lets us check whether we +need probN or probA. +

+In each inner() call, loc_h is the location of the root of this +dependency structure. +

DONE [#A] test and debug my brilliant idea
+CLOSED: 2008-06-08 Sun 10:28
+
DONE implement my brilliant idea.
+CLOSED: 2008-06-01 Sun 17:19
+e(sti) in dmv.py + +
DONE [#A] test inner() on sentences with duplicate words
+Works with eg. the sentence "h h h" +

+ +

6 Expectation Maximation in IO/DMV-terms

inner(s,t,LHS) calculates the expected number of trees headed by LHS from s to t (sentence positions). This uses the P_STOP and P_CHOOSE @@ -418,9 +413,9 @@ Since "adjacency" is not captured in regular CNF rules, we need two probabilites for each rule, and inner() has to know when to use which.

TODO Corpus access
+
TODO Corpus access
TOGROK sentences or rules as the "outer loop"? OPTIMIZE
+
TOGROK sentences or rules as the "outer loop"? OPTIMIZE
In regard to the E/M-step, finding P_STOP, P_CHOOSE. @@ -430,9 +425,9 @@ In regard to the E/M-step, finding P_STOP, P_CHOOSE.

8 Python-stuff

7 Python-stuff

@@ -455,9 +450,9 @@ In regard to the E/M-step, finding P_STOP, P_CHOOSE.

9 Git

8 Git

Setting up a new project:

@@ -522,6 +517,6 @@ Good tutorial:
  Author: Kevin Brubeck Unhammer
 <K.BrubeckUnhammer at student uva nl >
 
- Date: 2008/06/08 10:19:20
+ Date: 2008/06/08 12:43:26
 
Skrive vha. emacs + org-mode
 
diff --git a/DMVCCM.org b/DMVCCM.org
index 4d39ec9..b0bd3cd 100755
--- a/DMVCCM.org
+++ b/DMVCCM.org
@@ -17,53 +17,9 @@ But absolute, extended, really-quite-dead-now deadline: August
 - [[file:src/dmv.py][dmv.py]]
 - [[file:src/io.py][io.py]]
 - [[file:src/harmonic.py::harmonic%20py%20initialization%20for%20dmv][harmonic.py]]
-* TODO Adjacency and combining it with inner()
-Each DMV_Rule now has both a probN and a probA, for
-adjacencies. inner() needs the correct one in each case.
 
-Adjacency gives a problem with duplicate words/tags, eg. in the
-sentence "a a b". If this has the dependency structure b->a_{0}->a_{1},
-then b is non-adjacent to a_{0} and should use probN (for the LRStop and
-the attachment of a_{0}), while the other rules should all use
-probA. But within the e(0,2,b) we can't just say "oh, a has index 0
-so it's not adjacent to 2", since there's also an a at index 1, and
-there's also a dependency structure b->a_{1}->a_{0} for that. We want
-both. And in possibly much more complex versions.
 
-Ideas:
-- I first thought of decorating the individual words/tags in a
-  sentence with their indices, and perhaps just duplicating the
-  relevant rules (one for each index of the duplicate tags). But this
-  gives an explosion in attachment rules (although a contained
-  explosion, within the rules used in a sentence; but most sentences
-  will have at least two NN's so it will be a problem).
-- Then, I had a /brilliant/ idea. Just let e(), the helper function of
-  inner(), parametrize for an extra pair of boolean values for whether
-  or not we've attached anything to the left or right yet ("yet"
-  meaning "below"). So now, e() has a chart of the form [s, t, LHS,
-  Lattach, Rattach], and of course e(s,t,LHS) is the sum of the four
-  possible values for (Lattach,Rattach). This makes e() lots more
-  complex and DMV-specific though, so it's been rewritten in
-  inner_dmv() in dmv.py.
-- But, the problem is that we still need to be able get inner
-  probabilites for only trees where the h at location loc_h is the
-  root of the dependency. The previous idea does not give that. So I
-  made /another/ version of inner() that parametrizes for loc_h in the
-  chart (instead of Lattach and Rattach). loc_h also lets us check
-  whether we need probN or probA, in a much simpler fashion than the
-  Lattach / Rattach version.
-** TODO document this adjacency stuff better
-** TODO [#A] test and debug my brilliant idea
-*** Works on simple 'h h' sentence, but not with P_STOP, why?
-** DONE implement my brilliant idea.
-    CLOSED: [2008-06-01 Sun 17:19]
-[[file:src/dmv.py::def%20e%20s%20t%20LHS%20Lattach%20Rattach][e(sti) in dmv.py]]
-
-** DONE [#A] test inner() on sentences with duplicate words
-Works with eg. the sentence "h h h"
-
-
-* TODO [#A] P_STOP for IO/EM
+* TODO [#A] P_STOP and P_CHOOSE for IO/EM
 [[file:src/dmv.py::DMV%20probabilities][dmv-P_STOP]]
 Remember: The P_{STOP} formula is upside-down (left-to-right also).
 (In the article..not the thesis)
@@ -74,31 +30,53 @@ have to be updated along with the other P_{STOP} updates:
 - b[(RBAR, n_{h}), 'h'] = h_.probA  # h_ is RBAR stop rule
 - b[(LRBAR, n_{h}), 'h'] = h_.probA * _ h_.probA
 
-** How is the P_STOP formula different given other values for dir and adj?
+** P_STOP formulas for various dir and adj:
 Assuming this:
 - P_{STOP}(STOP|h,L,non_adj) = \sum_{corpus} \sum_{sloc(h)}
-  inner(s,t,(LRBAR,h)...) / \sum_{corpus} \sum_{s} \sum_{t>loc(h)} inner(s,t,(RBAR,h)...)
+  inner(s,t,_h_, ...) / \sum_{corpus} \sum_{s} \sum_{t>loc(h)}
+  inner(s,t,h_, ...)
 - P_{STOP}(STOP|h,R,adj) = \sum_{corpus} \sum_{s} \sum_{t=loc(h)}
-  inner(s,t,(LRBAR,h)...) / \sum_{corpus} \sum_{s} \sum_{t=loc(h)} inner(s,t,(RBAR,h)...)
-
-
+  inner(s,t,_h_, ...) / \sum_{corpus} \sum_{s} \sum_{t=loc(h)}
+  inner(s,t,h_, ...)
 
 (And P_{STOP}(-STOP|...) = 1 - P_{STOP}(STOP|...) )
-* TODO P_CHOOSE for IO/EM
-Write the formulas! should be easy?
+** P_CHOOSE formula:
+Assuming this:
+- P_{CHOOSE}(a|h,L) = \sum_{corpus} \sum_{s= loc(h)?
+- P_{CHOOSE}(a|h,R) = \sum_{corpus} \sum_{s=loc(h)} \sum_{r>s}
+  inner(s,r,_a_,...) / \sum_{corpus} \sum_{s=loc(h)} \sum_{t}
+  inner(s,t,h, ...)
+  - or s => loc(h)?
+
+
+* TOGROK Find Most Probable Parse of given test sentence
+One way of finding the MPP is to construct all complete trees, so
+could this be used with P_STOP to make sure we only get counts from
+complete trees? 
+
+(But then, there are probably MPP algorithms that avoid computing _all_
+complete trees for a sentence...)
+* TOGROK Combine the models
+* TODO Get a dependency parsed version of WSJ to test with
 * Initialization   
 [[file:~/Documents/Skole/V08/Probability/dmvccm/src/dmv.py::Initialization%20todo][dmv-inits]]
 
 We do have to go through the corpus, since the probabilities are based
 on how far away in the sentence arguments are from their heads.
-** TODO Separate initialization to another file?		      :PRETTIER:
-(It's rather messy.)
 ** TOGROK CCM Initialization    
 P_{SPLIT} used here... how, again?
+** DONE Separate initialization to another file?		      :PRETTIER:
+   CLOSED: [2008-06-08 Sun 12:51]
+[[file:src/harmonic.py::harmonic%20py%20initialization%20for%20dmv][harmonic.py]]
 ** DONE DMV Initialization probabilities
 (from initialization frequency)
 ** DONE DMV Initialization frequencies    
@@ -170,22 +148,6 @@ just a frequency count of the corpus...
 * [#C] Deferred
 ** TODO if loc_h == t, no need to try right-attachment rules	      :OPTIMIZE:
 ** TODO if loc_h == s, no need to try left-attachment rules 	      :OPTIMIZE:
-** DONE inner_dmv() should disregard rules with heads not in sent     :OPTIMIZE:
-   CLOSED: [2008-06-08 Sun 10:18]
-If the sentence is "nn vbd det nn", we should not even look at rules
-where
-: rule.head() not in "nn vbd det nn".split()
-This is ruled out by getting rules from g.rules(LHS, sent).
-
-Also, we optimize this further by saying we don't even recurse into
-attachment rules where
-: rule.head() not in sent[ s :r+1]
-: rule.head() not in sent[r+1:t+1]
-meaning, if we're looking at the span "vbd det", we only use
-attachment rules where both daughters are members of ['vbd','det']
-(although we don't (yet) care about removing rules that rewrite to the
-same tag if there are no duplicate tags in the span, etc., that would
-be a lot of trouble for little potential gain).
 ** TODO when reestimating P_STOP etc, remove rules with p < epsilon   :OPTIMIZE:
 ** TODO inner_dmv, short ranges and impossible attachment	      :OPTIMIZE:
 If s-t <= 2, there can be only one attachment below, so don't recurse
@@ -223,6 +185,51 @@ Need to clean up the representation.
 Stick with tag-strings in initialization then switch to numbers for
 IO-algorithm perhaps? Can probably afford more string-matching in
 initialization..
+** DONE inner_dmv() should disregard rules with heads not in sent     :OPTIMIZE:
+   CLOSED: [2008-06-08 Sun 10:18]
+If the sentence is "nn vbd det nn", we should not even look at rules
+where
+: rule.head() not in "nn vbd det nn".split()
+This is ruled out by getting rules from g.rules(LHS, sent).
+
+Also, we optimize this further by saying we don't even recurse into
+attachment rules where
+: rule.head() not in sent[ s :r+1]
+: rule.head() not in sent[r+1:t+1]
+meaning, if we're looking at the span "vbd det", we only use
+attachment rules where both daughters are members of ['vbd','det']
+(although we don't (yet) care about removing rules that rewrite to the
+same tag if there are no duplicate tags in the span, etc., that would
+be a lot of trouble for little potential gain).
+* Adjacency and combining it with inner()
+Each DMV_Rule now has both a probN and a probA, for
+adjacencies. inner() needs the correct one in each case.
+
+Adjacency gives a problem with duplicate words/tags, eg. in the
+sentence "a a b". If this has the dependency structure b->a_{0}->a_{1},
+then b is non-adjacent to a_{0} and should use probN (for the LRStop and
+the attachment of a_{0}), while the other rules should all use
+probA. But within the e(0,2,b) we can't just say "oh, a has index 0
+so it's not adjacent to 2", since there's also an a at index 1, and
+there's also a dependency structure b->a_{1}->a_{0} for that. We want
+both. And in possibly much more complex versions.
+
+So now we call inner() for each location of a head, and on each
+terminal, loc_h must equal s (and t). In the recursive attachment
+calls, we use the locations (sentence indices) of words to the left or
+right of the head in calls to inner(). loc_h lets us check whether we
+need probN or probA.
+
+In each inner() call, loc_h is the location of the root of this
+dependency structure.
+** Possible alternate type of adjacency
+K&M's adjacency is just whether or not an argument has been generated
+in the current direction yet. One could also make a stronger type of
+adjacency, where h and a are not adjacent if b is in between, eg. with
+the sentence "a b h" and the structure ((h->a), (a->b)), h is
+K&M-adjacent to a, but not next to a, since b is in between. It's easy
+to check this type of adjacency in inner(), but it needs new rules for
+P_STOP reestimation.
 * Expectation Maximation in IO/DMV-terms
 inner(s,t,LHS) calculates the expected number of trees headed by LHS
 from s to t (sentence positions). This uses the P_STOP and P_CHOOSE
diff --git a/src/dmv.py b/src/dmv.py
index eb81c4b..ad3ec51 100755
--- a/src/dmv.py
+++ b/src/dmv.py
@@ -230,13 +230,14 @@ class DMV_Rule(io.CNF_Rule):
 ###################################
 # dmv-specific version of inner() #
 ###################################
-def locs(h, sent, s=0, t=None):
+def locs(h, sent, s=0, t=None, remove=None):
     '''Return the locations of h in sent, or some fragment of sent (in the
     latter case we make sure to offset the locations correctly so that
     for any x in the returned list, sent[x]==h).'''
     if t == None:
         t = len(sent)
-    return [i+s for i,w in enumerate(sent[s:t]) if w == h]
+    return [i+s for i,w in enumerate(sent[s:t])
+            if w == h and not (i+s) == remove]
 
 
 def inner_dmv(s, t, LHS, loc_h, g, sent, chart):
@@ -301,12 +302,14 @@ def inner_dmv(s, t, LHS, loc_h, g, sent, chart):
 
                     else: # not a STOP, an attachment rewrite:
                         for r in range(s, t):
+                            # if loc_h == t, no need to try right-attachments,
+                            # if loc_h == s, no need to try left-attachments... todo
                             p_h = rule.p_ATTACH(r, loc_h, s=s)
                             if rule.LHS() == L: 
                                 locs_L = [loc_h]
-                                locs_R = locs(head(R), sent_nums, r+1, t+1)
+                                locs_R = locs(head(R), sent_nums, r+1, t+1, loc_h)
                             elif rule.LHS() == R:
-                                locs_L = locs(head(L), sent_nums,  s,  r+1)
+                                locs_L = locs(head(L), sent_nums,  s,  r+1, loc_h)
                                 locs_R = [loc_h]
                             # see http://tinyurl.com/4ffhhw 
                             p += sum([e(s, r, L, loc_L, n_t+1) *
@@ -431,13 +434,10 @@ P_STOP(-STOP|...) = 1 - P_STOP(STOP|...)
         # have to go through _all_ places where h appears in the
         # sentence...how? how to make sure it _works_?
         chart = {} 
-        inner_sent_dmv(sent, g, chart) #todo, current. Later on,
-                                       #generalize to all heads, since
-                                       #we now have the chart of all
-                                       #of them anyway
         locs_h = locs(h_tag, sent)
         io.debug( "locs_h:%s, sent:%s"%(locs_h,sent) , 2)
         for loc_h in locs_h:
+            inner_dmv(0, len(sent)-1, ROOT, loc_h, g, sent, chart)
             for s in range(loc_h): # s{AffOT7eZ45VHLL{Uh3%D^?ZW%?^vh^|s
zv@CHdJfM-BL=rSHG4Y8f(?5U&UyQ++7~+GznDC&+2N9o?NId7ZR!j^{&u6FSd(Qcu
z@A-btci)xPRzu|{KKHXv4^5ti>ZgunjHR%8EA*uk!K{b3)LTDCFkE(HFDmmuyOrBs
zQ-yhi48m*+&^O9HtfwY*j91Gi)wi1i8fS2^=q%%sZJB;+*__hAQlN-XWK^Mj4Pjzt
z7fdTq(xK#pB7XtSDUfdmEb8L6q395UH($>d4XVb#;s7AaHI0t9{)8^t6ApPS*=#AZ
zkYCQw$#9}g_6LU#{TRj5PS!mvJ{DC9dQ5l2`GDyF3n8n(IUVvB08Ndyv~{6QN7|Bs
z_h8C&!hy*g^4SaoHmbM;Xl}`nbv2i{om=cGAw8&go
z?zwO|^(w0x{m^-&IgabF1AU55b>j)vf|!-oWsxRgBbcB!V;RqJN%4}vF?tf~_Dgii
zHXai=K)qdC*8eVBG)b(lUX3fE(I)hlvt4!A_L?{etx`#bm|n|PNkJdP_uwn#Z{u4L
zSIWw^j}T|+fiQrY*+Y?q=6q@S^uYMx5n3%bPB?Ha^l-zQ3
zZXsV7x4GmI#${D7nQ3sk+s3zh>+1WU66nf4B7VnP+YzyxFy
z-^H4fL$;#j6st|$RxG7`u(G~ts|DNl`xlhz{@I)td{C?tr!#78|BB6W2_?8JS#;Q!
z&RjeUSdG=`>)ws-T8#W`S=GTNWhH^};sj1H{jY0tE$a(>hM3ihWgfROOfBAKIR#Vc
z#SWONrgzyx>3qJi{L3`poCEURwJO|Iw+c2s--e3iXinq#<1sTmJZj$ZiN33HLrXkhSE(|T@5
zBbRu%;Ab>5R%AJCMshj2I~doi>i2^;>8rsFbbZK2#i2u(rZ0v@VrJewm%D78&F1DK
z*)!}H$rd7W=P%7&E<_5M)A^hkxs+crd&o$3;wjpm9M=y@gVJ;*+1F^#oZU6AfV`in
z3A&fug%9Y@*O*Jm#-N3oHuc==I@o|Mb7n0tI^CXr>3lL3~V)
zhW9oFq4FqMnS!;HVaP7bS=oG!Mo0P^^AfQ`V5h(?fs6Fc$nNC++QrWZEDKx`SgBQf
zMIa_{RbZ-8(jOxojkBW8*NU|SiqtncfhJuX-J}HEbZ2yPD|-358?eD^s78Y^W<-q+
L9~x{m_}|Lk3&v?h

delta 1711
zcwU86O>7%g5T4mxuf4W6cG5UW6XN{Wj+?|O`E%MjKTXmG+5$?T5p5FMW*g%EIDgwD
zEp}7M1x11bWss06Br0*K)Eho=q#TeCYEKB!0}?`&5UNyE^$3EQwd-=j%6xeq&wOv*
z%=cby`(d^&|J3wOetdLt0g9h8wh^}Vm78_nYXP{S+%Q|kM>D>TkB@24q<#Q3bP7iU69ow!+!wh6<7`c
zW;DpSB(l0O4ejWn8x4ty9hk9nI5L?=-p7FHs5^hZ;WrHbBbkb@;)08D7h8Gp|N32e(6o*1
z^m_BX@S?(mkafe>A!Ww}DW}jKgq*@B>Y{kdlkx*-d*wKpqBlnI)(-5Z+16+x$GHJW
zkMQmuNckbhVP5Cp9e}n8b{TM4+#aSB3O`sLsLpA0uXRnUQdj+yX!{mj`M=sG5GUwR
zB&HwX5~n6JX*P4t}aEnzw&Firc~JA;xqTM;6BT+!kgHSmGov@R+$@q8KmaRBj>b%j2|=rR4)=NoioQ(*R)MOKy%=
zgfPQnT3B9NxxC((f6#qVrHS5mj~OyXkHD^8FF!=yzF&0MsH2Vnbc3$y7}Nq-Eo(H{s@Q5-@)2xxuve=_
z6T@_;Z=y;CJ_e7`@;bhzY8r^qqk#aO9LUk*
zfzvod#}i3yb9wCoeU*sd96d~o=}xE4P|IL<$hjG(N)3_+Z)%K|2KVEA`f{)ZG5_7*
z4+>tSPY6@g1+B*%iMUStcb$z3BmS&dKVqvR;o(EVh!^3D=HO7?`(
z67ll$85O2wQ|Hiugfmg+x;q2DEU+Q4S*q}gKvdwWz==W$w}+ZT^Riql6}K+%9{n^l
lj%Vqm;T9!e(CBbmof5-xEcY9#QErSGO-8ec2Kw28_%CGGLCpXF

diff --git a/src/io.pyc b/src/io.pyc
index a79c403997f05df38bd2bac52913a7d73041b59c..10b2c64a8853e9c931829e61eb43a33c3f276ad5 100644
GIT binary patch
delta 1166
zcwSwT%WD%+6o=29WG2%*8k8h6lcq^u+O$d`7Nj)IwLM`G(5QTtHT!?EGm*T>WOR<9e3%uv%WgWO*=HBy|-#O=|n)s54R1Z8(
z2VO`nK>cy@b(XJEMFPm~M1o6}4YnAhZD85K>H1U*(j5CzFlGx4zdL29Ls7?
z+4Sl?Hf?qA1Q|tyQZ#;Nr|lAC85&YAcb$;NIM=0_>ZYRXS`)73H%eE3T3
zWcKPu^*z&6%0e}nf~lw^B;c-dH%Khoa0d_tTjVf4kRR;nvr%;F-lc?M|i$yNdgs5myh{o7r`lvRuJG
zo(bU&`fwu<@%K{@m0F@ixqPf}%Z%acz!_QGRNN0t;E*;Pq?G1h
zv&bRBaY7t7v;lvtc^D`33p4I%QRcxPS`7VqyodDV=quUBkLcN(%zpkv&#y@{A!K7#
ze;ylw`t)+5=GInR?!M+$T!jRV8r>=3D9k$H6oFLRpKoxrptiF(OTxt}Q}E
zBq~A=QXU>G1(Du72?3#05$_dy77w00c@R$?oi|Anio?#g`+nw|@4cD*ny#f2wLbxU
z|NT^E5gJ41^BSM4YZ8F7m54uj0+927!=GMoyui&WznIhk@mq;!enL+MO(k@13#<|Gx@!NCPQ
zklu#K=PKlME5+pk9%>!9>j^Wpw&zJpQHr=84%y6g%gf)kOrWByJY_O&DjQ6$X{yEy
z5i|&b*r~<3D9=?nly6n;-^;JsCny#mXu}z8F;3LB-f|MObFct^`=iWL`>B0mCNa1q
z(iWJtBq0Jn=x^c_v!ettS|>H-#rXzNd||Bjs7Fze=aR^BrLQ*;xML26
zhnmh|f&mf6Z>GiD?un+nqy$HzXZuNEk@hKUzoM}>QTTE|DK5n*&c^z(qCFzpBuEp8---5K5Dl@%2rfe7&si&