Sync ctags with upstream so that most parsers can be copied from uctags (#2018)
commita1cf475fcf2c81420bfa90307585ec645cfb4dc1
authorJiří Techet <techet@gmail.com>
Sat, 6 Apr 2019 02:14:30 +0000 (6 04:14 +0200)
committerelextr <elextr@gmail.com>
Sat, 6 Apr 2019 02:14:30 +0000 (6 12:14 +1000)
tree003f5b3220bc23705a3b00d36226b8de09c0da88
parentb8cdfddf985f666ec09b79237f5ae19fc02a9e9a
Sync ctags with upstream so that most parsers can be copied from uctags (#2018)

* Use latest version of htable

* Use latest version of mio

* Use latest version of objpool

* Use latest version of ptrarray

* Use latest version of vstring

This also requires adding trashbox.c/h which is now used by vstring and
inline macros from inline.h.

* Rename fieldSpec to fieldDefinition

See b56bd065123d69087acd6f202499d71a86a7ea7a upstream.

* Rename kindOption to kindDefinition

See e112e8ab6e0933b5bd7922e0dfb969b1f28c60fa upstream

* Rename kinds field in parserDefinition to kindTable

See 09ae690face8b5cde940e2d7cf40f8860381067b upstream.

* Rename structure fields about field in parserDefinition

See a739fa5fb790bc349a66b2bee0bf42cf289994e8 upstream.

* Use kindIndex instead of kindDefinition

This patch replaces kindDefinition related entries from sTagEntryInfo
with kindIndex so kinds are referenced indirectly using the index. For
more info please refer to commits:

16a2541c0698bd8ee03c1be8172ef3191f6e695a
f92e6bf2aeb21fd6b04756487f98d0eefa16d9ce

Some other changes had to be made to make the sources compile (without
bringing all the diffs from upstream). At some places, which aren't used
by Geany, only stub implementations have been created.

In particular, the regex parser has been disabled (for now?) because its
current implementation doesn't allow accessing kindDefinitions using
index and allowing this would require big changes in its implementation.
The affected parsers are Cobol, ActionScript and HTML. For HTML we can
use the token-based parser from upstream, and we should consider
whether Cobol and ActionScript are worth the effort to maintain a separate
regex implementation using GRegex (IMO these languages are dead enough
not to justify the extra effort).

The patch also disables tests for languages using regex parsers.

* Rename roleDesc to roleDefinition

See 1345725842c196cc0523ff60231192bcd588961b upstream. Since we don't care
about roles in Geany, we don't have to do the additional stuff the upstream
patch does.

* Add XTAG_ANONYMOUS used by jscript

See 0e4c5d4a0461bc8d9616fe3b97d75b91d014246e upstream.

* Include stdint.h in entry.h

* Don't use hash value as an Anonymous field identifier

Instead of something like "Anonymous0ab283cd9402" use sequential integer
values like "Anonymous1".

* Call anonReset in main part

See 3c91b1ea509df238feb86c9cbd552b621e462653 upstream.

* Use upstream javascript parser

* Use upstream css parser

* Create correctly sized MIO for 0 size

See https://github.com/universal-ctags/ctags/pull/1951

* Always enable promise API and subparsers for Geany

* Support subparsers in Geany and add HTML parser demonstrating this feature

This feature requires several changes:

1. Propagating language of the tag from ctags to Geany so we know whether
the tag comes from a master parser or a subparser.

2. We need to address the problem that tag types from a subparsers can
clash with tag types from master parsers or other subparsers used by the
master parser. For instance, HTML and both its css and javascript
subparsers use tm_tag_class_t but HTML uses it for <h2> headings, and
css and javascript for classes. Representing all of them using
tm_tag_class_t would lead to complete mess where all of these types would
for instance be listed in the same branch of the tree in the sidebar.

To avoid this problem, this patch adds another mapping for subparsers where
each tag type can be mapped to another tag type (which isn't used neither
by master parser or other subparsers). To avoid unwanted clashes with other
parsers, only tags explicitly mentioned in such mappings are added to tag
manager; other subparser tags are discarded.

For HTML this patch introduces mapping only for tm_tag_function_t (which
in this case maps to the same type) to mimick the previous HTML parser
behavior but other javascript and css tag types can be added this way
in the future too.

3. Since in most of the code Geany and tag manager assume that tags from
one file use the same language, subparser's tags are modified to have the
same language like the master parser.

4. HTML parser itself was copied from upstream without any modifications.
Tests were fixed as the parser now correctly ignores comments.

* Rename truncateLine field of tagEntryInfo

See 0e70b22791877322598f03ecbe3eb26a6b661001 upstream. Needed for Fortran
parser.

* Add dummy mbcs.h and trace.h

Included by javascript parser.

* Introduce an accessor to `locate' field of `Option'

See fb5ef68859f71ff2949f1d9a7cab7515f523532f upstream. Needed for Fortran.

* Add numarray.c/h

Needed by various parsers.

* Add getLanguageForFilename() and getLanguageForCommand()

See

416c5e6b8807feaec318d7f8addbb4107370c187
334e072f9d6d9954ebd3eb89bbceb252c20ae9dd

upstream. Needed for Sh parser.

* txt2tags: Fix scope separator definition and re-enable tests

* Rename rest.c to rst.c to match upstream filename

* Use upstream asciidoc and rst parsers

* Add asciidoc and rst unit tests

* Rename conf.c to iniconf.c to match upstream filename

* Add tests of conf, diff, md parsers from universal ctags

* Add more ctags unit tests

This patch adds unit tests for: nsis, docbook, haskell, haxe, abaqus, vala,
abc.

The only missing unit tests are for GLSL and Ferite parsers which
however share the implementation with the C parser and should be
reasonably well covered by other C-like language tests.

The tests were put together from various tutorials and help of the
languages in order to cover the tags these parsers generate. No guarantee
they'd compile with real parsers.

* Rename latex.c to tex.c to match upstream filename

* Rename entry points of parsers to match upstream names

* Initialize trashbox

* Add newline to the end of file
119 files changed:
ctags/Makefile.am
ctags/main/ctags-api.c
ctags/main/ctags-api.h
ctags/main/dependency.c
ctags/main/entry.c
ctags/main/entry.h
ctags/main/field.c
ctags/main/field.h
ctags/main/htable.c
ctags/main/htable.h
ctags/main/inline.h [new file with mode: 0644]
ctags/main/kind.c
ctags/main/kind.h
ctags/main/lcpp.c
ctags/main/lcpp.h
ctags/main/lregex.c
ctags/main/lxcmd.c
ctags/main/lxpath.c
ctags/main/mbcs.h [new file with mode: 0644]
ctags/main/mio.c
ctags/main/mio.h
ctags/main/numarray.c [new file with mode: 0644]
ctags/main/numarray.h [new file with mode: 0644]
ctags/main/objpool.c
ctags/main/objpool.h
ctags/main/options.c
ctags/main/options.h
ctags/main/parse.c
ctags/main/parse.h
ctags/main/parsers.h
ctags/main/promise.c
ctags/main/ptrarray.c
ctags/main/ptrarray.h
ctags/main/read.c
ctags/main/read.h
ctags/main/trace.h [new file with mode: 0644]
ctags/main/trashbox.c [new file with mode: 0644]
ctags/main/trashbox.h [new file with mode: 0644]
ctags/main/types.h
ctags/main/vstring.c
ctags/main/vstring.h
ctags/main/xtag.c
ctags/main/xtag.h
ctags/parsers/abaqus.c
ctags/parsers/abc.c
ctags/parsers/asciidoc.c
ctags/parsers/asm.c
ctags/parsers/basic.c
ctags/parsers/c.c
ctags/parsers/css.c
ctags/parsers/diff.c
ctags/parsers/docbook.c
ctags/parsers/erlang.c
ctags/parsers/fortran.c
ctags/parsers/go.c
ctags/parsers/haskell.c
ctags/parsers/haxe.c
ctags/parsers/html.c
ctags/parsers/iniconf.c [moved from ctags/parsers/conf.c with 91% similarity]
ctags/parsers/jscript.c
ctags/parsers/json.c
ctags/parsers/lua.c
ctags/parsers/make.c
ctags/parsers/markdown.c
ctags/parsers/matlab.c
ctags/parsers/nsis.c
ctags/parsers/objc.c
ctags/parsers/pascal.c
ctags/parsers/perl.c
ctags/parsers/php.c
ctags/parsers/powershell.c
ctags/parsers/python.c
ctags/parsers/r.c
ctags/parsers/rest.c [deleted file]
ctags/parsers/rst.c [new file with mode: 0644]
ctags/parsers/ruby.c
ctags/parsers/rust.c
ctags/parsers/sh.c
ctags/parsers/sql.c
ctags/parsers/tcl.c
ctags/parsers/tex.c [moved from ctags/parsers/latex.c with 95% similarity]
ctags/parsers/txt2tags.c
ctags/parsers/verilog.c
ctags/parsers/vhdl.c
src/tagmanager/tm_parser.c
src/tagmanager/tm_parser.h
src/tagmanager/tm_source_file.c
tests/ctags/Makefile.am
tests/ctags/complex-return.js.tags
tests/ctags/geany.nsi [new file with mode: 0644]
tests/ctags/geany.nsi.tags [new file with mode: 0644]
tests/ctags/js-class-related-unterminated.js.tags
tests/ctags/js-let.js.tags
tests/ctags/js-string-continuation.js.tags
tests/ctags/jsFunc_tutorial.js.tags
tests/ctags/simple.abc [new file with mode: 0644]
tests/ctags/simple.abc.tags [new file with mode: 0644]
tests/ctags/simple.asciidoc [new file with mode: 0644]
tests/ctags/simple.asciidoc.tags [new file with mode: 0644]
tests/ctags/simple.conf [new file with mode: 0644]
tests/ctags/simple.conf.tags [new file with mode: 0644]
tests/ctags/simple.diff [new file with mode: 0644]
tests/ctags/simple.diff.tags [new file with mode: 0644]
tests/ctags/simple.docbook [new file with mode: 0644]
tests/ctags/simple.docbook.tags [new file with mode: 0644]
tests/ctags/simple.hs [new file with mode: 0644]
tests/ctags/simple.hs.tags [new file with mode: 0644]
tests/ctags/simple.html.tags
tests/ctags/simple.hx [new file with mode: 0644]
tests/ctags/simple.hx.tags [new file with mode: 0644]
tests/ctags/simple.inp [new file with mode: 0644]
tests/ctags/simple.inp.tags [new file with mode: 0644]
tests/ctags/simple.js.tags
tests/ctags/simple.md [new file with mode: 0644]
tests/ctags/simple.md.tags [new file with mode: 0644]
tests/ctags/simple.rst [new file with mode: 0644]
tests/ctags/simple.rst.tags [new file with mode: 0644]
tests/ctags/simple.vala [new file with mode: 0644]
tests/ctags/simple.vala.tags [new file with mode: 0644]