forked from 0ad/0ad
Remove contrib JS-lexer and use upstream one
As we depend on Babel for i18n tooling nowadays, we can also use its maintained JavaScript-lexer, instead of the one forked from an old version of Babel. Doing so reduces the amount of code we have to maintain, adds new functionality (like properly handling hexadecimal unicode notations) and fixes some regex related incompatibilities which makes generating portable object templates possible with recent Python versions. The use of `tokenize(…, dotted=False)` is necessary, as we use this lexer for C++ code as well and detection of most strings in C++ code fails otherwise. As dotted names weren't supported by version of the lexer we used until now, this isn't a regression. Patch by: @Dunedan Accepted by: @Stan Differential Revision: https://code.wildfiregames.com/D5293 This was SVN commit r28124.
This commit is contained in:
parent
fa1dc98103
commit
53e00e1953
@ -1,6 +1,6 @@
|
||||
#!/usr/bin/env python3
|
||||
#
|
||||
# Copyright (C) 2022 Wildfire Games.
|
||||
# Copyright (C) 2024 Wildfire Games.
|
||||
# All rights reserved.
|
||||
#
|
||||
# Redistribution and use in source and binary forms, with or without modification, are permitted provided that the
|
||||
@ -110,7 +110,7 @@ class javascript(Extractor):
|
||||
|
||||
def extractJavascriptFromFile(self, fileObject):
|
||||
|
||||
from extractors.jslexer import tokenize, unquote_string
|
||||
from babel.messages.jslexer import tokenize, unquote_string
|
||||
funcname = message_lineno = None
|
||||
messages = []
|
||||
last_argument = None
|
||||
@ -121,7 +121,7 @@ class javascript(Extractor):
|
||||
comment_tags = self.options.get('commentTags', [])
|
||||
keywords = self.options.get('keywords', {}).keys()
|
||||
|
||||
for token in tokenize(fileObject.read()):
|
||||
for token in tokenize(fileObject.read(), dotted=False):
|
||||
if token.type == 'operator' and \
|
||||
(token.value == '(' or (call_stack != -1 and \
|
||||
(token.value == '[' or token.value == '{'))):
|
||||
|
@ -1,2 +1,2 @@
|
||||
babel~=2.6
|
||||
babel~=2.12
|
||||
lxml~=4.5
|
||||
|
Loading…
Reference in New Issue
Block a user