Update search crawler for python3
commit2c84a8ec63d14738e5db65083940441a87c0e248
authorMagnus Hagander <magnus@hagander.net>
Wed, 23 Jan 2019 21:14:56 +0000 (23 22:14 +0100)
committerMagnus Hagander <magnus@hagander.net>
Sat, 26 Jan 2019 15:19:26 +0000 (26 16:19 +0100)
tree09a5cb0ef8e54fd1dc5861cf14d92d320be4d1ca
parentc350fa47f10f9a2ce45ef42dc64129bf941eb2c7
Update search crawler for python3

This includes the switch to requests, but also a bunch of other changes.

While at it, fix it so we can do proper https validation.
tools/search/crawler/lib/basecrawler.py
tools/search/crawler/lib/parsers.py
tools/search/crawler/lib/sitemapsite.py