Handle licence expressions such as MIT/Apache-2.0

[anthonyharrison]

Some package metadata contains expressions of the form MIT/Apache-2.0. Whilst this is not a valid SPDX licence expression, it could be converted to a valid SPDX expression MIT OR Apache-2.0

Currently this expression results in an exception being raised.

>>> expression = "MIT/Apache-2.0"
>>> parsed = licensing.parse(expression)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/data/Documents/licence_expressions/lib/python3.10/site-packages/license_expression/__init__.py", line 540, in parse
    tokens = list(self.tokenize(
  File "/data/Documents/licence_expressions/lib/python3.10/site-packages/license_expression/__init__.py", line 604, in tokenize
    for token in tokens:
  File "/data/Documents/licence_expressions/lib/python3.10/site-packages/license_expression/__init__.py", line 997, in replace_with_subexpression_by_license_symbol
    for token_group in token_groups:
  File "/data/Documents/licence_expressions/lib/python3.10/site-packages/license_expression/__init__.py", line 936, in build_token_groups_for_with_subexpression
    tokens = list(tokens)
  File "/data/Documents/licence_expressions/lib/python3.10/site-packages/license_expression/__init__.py", line 598, in <genexpr>
    tokens = (t for t in tokens if t.string and t.string.strip())
  File "/data/Documents/licence_expressions/lib/python3.10/site-packages/license_expression/__init__.py", line 922, in build_symbols_from_unknown_tokens
    for symtok in build_token_with_symbol():
  File "/data/Documents/licence_expressions/lib/python3.10/site-packages/license_expression/__init__.py", line 902, in build_token_with_symbol
    toksym = LicenseSymbol(string)
  File "/data/Documents/licence_expressions/lib/python3.10/site-packages/license_expression/__init__.py", line 1214, in __init__
    raise ExpressionError(
license_expression.ExpressionError: Invalid license key: the valid characters are: letters and numbers, underscore, dot, colon or hyphen signs and spaces: 'MIT/Apache-2.0'

[AyanSinhaMahapatra]

@anthonyharrison the license-expression library is used mostly to parse valid license expressions with explicit AND, OR, WITH. There are a lot of ecosystem specific or other types of commonly used license expressions which are handled in the scancode license detection and extracted license statement parsing, or through license rules. This is since it becomes increasingly difficult to support all different kinds of license expressions out in the wild without any specification.

Scancode does detect this correctly:

"detected_license_expression": "mit OR apache-2.0",
      "detected_license_expression_spdx": "MIT OR Apache-2.0",
      "license_detections": [
        {
          "license_expression": "mit OR apache-2.0",
          "license_expression_spdx": "MIT OR Apache-2.0",
          "matches": [
            {
              "license_expression": "mit OR apache-2.0",
              "license_expression_spdx": "MIT OR Apache-2.0",
              "from_file": "test",
              "start_line": 1,
              "end_line": 1,
              "matcher": "1-hash",
              "score": 99.0,
              "matched_length": 4,
              "match_coverage": 100.0,
              "rule_relevance": 99,
              "rule_identifier": "mit_or_apache-2.0_1.RULE",
              "rule_url": "https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/rules/mit_or_apache-2.0_1.RULE",
              "matched_text": " MIT/Apache-2.0",
              "matched_text_diagnostics": "MIT/Apache-2.0"
            }
          ],
          "identifier": "mit_or_apache_2_0-48698550-8e33-0b55-dc83-ac89d00625dd"
        }
      ],