Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No installation instructions #24

Open
jacobjove opened this issue Mar 29, 2021 · 26 comments
Open

No installation instructions #24

jacobjove opened this issue Mar 29, 2021 · 26 comments

Comments

@jacobjove
Copy link
Contributor

jacobjove commented Mar 29, 2021

Can you please add (1) prereqs and (2) installation instructions to the README?

I assumed I could use poetry add --dev actions-includes. However, when I tried this, I got a SolverProblemError "Because actions-includes (0.0.post117) requires Python >=3.8." (I'm still using Python 3.7, so maybe I can't use actions-includes.)

@mithro
Copy link
Owner

mithro commented Mar 29, 2021

Maybe you could use the docker image?

actions-includes/Makefile

Lines 130 to 140 in dc06f82

image-test: image
GITHUB_REPOSITORY=mithro/actions-includes \
GITHUB_SHA=$$(git rev-parse HEAD) \
docker run \
--workdir /github/workspace \
--rm \
-e GITHUB_SHA \
-e GITHUB_REPOSITORY \
-v "$$PWD":"/github/workspace" \
actions-includes \
.github/workflows/local.yml

@jacobjove
Copy link
Contributor Author

jacobjove commented Mar 29, 2021

Sorry, I'm not sure exactly what it would look like to "use the docker image"...

I think this is a really cool project, by the way.

What I actually want to do is make a pre-commit hook in my project (using https://pre-commit.com/) to preprocess the yaml with actions-includes. Do you think that would be possible?

@mithro
Copy link
Owner

mithro commented Mar 29, 2021

If you have docker installed, you should be able to do something like; docker run -it --entrypoint="" ghcr.io/mithro/actions-includes/image:main python -m actions_includes --help ?

@jacobjove
Copy link
Contributor Author

jacobjove commented Mar 30, 2021

I added the following to my docker-compose.yml:

  github_workflow_preprocessor:
    command: python -m actions_includes ./.github/workflows-src/integration.yml ./.github/workflows/integration.yml
    entrypoint: ''
    image: ghcr.io/mithro/actions-includes/image:main
    user: root
    volumes:
      - .:/github/workspace

But, when I ran docker-compose up github_workflow_preprocessor, I got the following error:

github_workflow_preprocessor_1  | Traceback (most recent call last):
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/runpy.py", line 197, in _run_module_as_main
github_workflow_preprocessor_1  |     return _run_code(code, main_globals, None,
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/runpy.py", line 87, in _run_code
github_workflow_preprocessor_1  |     exec(code, run_globals)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/__main__.py", line 25, in <module>
github_workflow_preprocessor_1  |     sys.exit(main())
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 998, in main
github_workflow_preprocessor_1  |     git_root_output = subprocess.check_output(
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/subprocess.py", line 424, in check_output
github_workflow_preprocessor_1  |     return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/subprocess.py", line 505, in run
github_workflow_preprocessor_1  |     with Popen(*popenargs, **kwargs) as process:
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/subprocess.py", line 951, in __init__
github_workflow_preprocessor_1  |     self._execute_child(args, executable, preexec_fn, close_fds,
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/subprocess.py", line 1823, in _execute_child
github_workflow_preprocessor_1  |     raise child_exception_type(errno_num, err_msg, err_filename)
github_workflow_preprocessor_1  | FileNotFoundError: [Errno 2] No such file or directory: 'git'

I see that the error comes from here:

git_root_output = subprocess.check_output(

But, I'm not sure what exactly is happening.

@jacobjove
Copy link
Contributor Author

jacobjove commented Mar 30, 2021

@mithro, how do you expect for actions_includes to normally be used? (Do you expect it to be installed via pip? If so, is it only available to people using Python 3.8+?)

@mithro
Copy link
Owner

mithro commented Mar 30, 2021

@icobfred - That looks like an issue with my docker image not containing the git command....

@mithro
Copy link
Owner

mithro commented Mar 30, 2021

FYI - @iacobfred, actions_includes uses the git command to figure out the top level directory.

@mithro
Copy link
Owner

mithro commented Mar 30, 2021

Can you give it another go now that git should be in the docker image? docker pull ghcr.io/mithro/actions-includes/image:sha-27715a2

@jacobjove
Copy link
Contributor Author

jacobjove commented Mar 30, 2021

@mithro, I pulled the updated image and tried again. Closer this time! But I got another error:

Recreating modularhistory_github_workflow_preprocessor_1 ... done
Attaching to modularhistory_github_workflow_preprocessor_1
github_workflow_preprocessor_1  | Expanding ./.github/workflows-src/integration.yml into ./.github/tmp/integration.yml
github_workflow_preprocessor_1  | Expanding workflow file from: /github/workspace/.github/workflows-src/integration.yml
github_workflow_preprocessor_1  |                           to: .github/tmp/integration.yml
github_workflow_preprocessor_1  | Loading yaml file /github/workspace/.github/workflows-src/integration.yml with contents md5 of 2b45ab17f42d5a42b1d7be32938056bc
github_workflow_preprocessor_1  | Traceback (most recent call last):
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/runpy.py", line 197, in _run_module_as_main
github_workflow_preprocessor_1  |     return _run_code(code, main_globals, None,
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/runpy.py", line 87, in _run_code
github_workflow_preprocessor_1  |     exec(code, run_globals)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/__main__.py", line 25, in <module>
github_workflow_preprocessor_1  |     sys.exit(main())
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 1031, in main
github_workflow_preprocessor_1  |     out_data = expand_workflow(current_action, to_path, insert_check)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 916, in expand_workflow
github_workflow_preprocessor_1  |     data = yaml_load(current_workflow, '\n'.join(workflow_data))
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 817, in yaml_load
github_workflow_preprocessor_1  |     return yaml.load(yaml_data, Loader=RoundTripLoaderWithExp)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/main.py", line 1057, in load
github_workflow_preprocessor_1  |     return loader._constructor.get_single_data()
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 111, in get_single_data
github_workflow_preprocessor_1  |     return self.construct_document(node)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 121, in construct_document
github_workflow_preprocessor_1  |     for _dummy in generator:
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 1533, in construct_yaml_map
github_workflow_preprocessor_1  |     self.construct_mapping(node, data, deep=True)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 730, in construct_mapping
github_workflow_preprocessor_1  |     return RoundTripConstructor.construct_mapping(self, node, maptyp, deep)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 1447, in construct_mapping
github_workflow_preprocessor_1  |     value = self.construct_object(value_node, deep=deep)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 144, in construct_object
github_workflow_preprocessor_1  |     data = self.construct_non_recursive_object(node)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 186, in construct_non_recursive_object
github_workflow_preprocessor_1  |     for _dummy in generator:
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 1533, in construct_yaml_map
github_workflow_preprocessor_1  |     self.construct_mapping(node, data, deep=True)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 730, in construct_mapping
github_workflow_preprocessor_1  |     return RoundTripConstructor.construct_mapping(self, node, maptyp, deep)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 1447, in construct_mapping
github_workflow_preprocessor_1  |     value = self.construct_object(value_node, deep=deep)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 144, in construct_object
github_workflow_preprocessor_1  |     data = self.construct_non_recursive_object(node)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 186, in construct_non_recursive_object
github_workflow_preprocessor_1  |     for _dummy in generator:
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 1533, in construct_yaml_map
github_workflow_preprocessor_1  |     self.construct_mapping(node, data, deep=True)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 730, in construct_mapping
github_workflow_preprocessor_1  |     return RoundTripConstructor.construct_mapping(self, node, maptyp, deep)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 1447, in construct_mapping
github_workflow_preprocessor_1  |     value = self.construct_object(value_node, deep=deep)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 144, in construct_object
github_workflow_preprocessor_1  |     data = self.construct_non_recursive_object(node)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 186, in construct_non_recursive_object
github_workflow_preprocessor_1  |     for _dummy in generator:
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 1525, in construct_yaml_seq
github_workflow_preprocessor_1  |     data.extend(self.construct_rt_sequence(node, data))
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 1297, in construct_rt_sequence
github_workflow_preprocessor_1  |     ret_val.append(self.construct_object(child, deep=deep))
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 144, in construct_object
github_workflow_preprocessor_1  |     data = self.construct_non_recursive_object(node)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 186, in construct_non_recursive_object
github_workflow_preprocessor_1  |     for _dummy in generator:
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 1533, in construct_yaml_map
github_workflow_preprocessor_1  |     self.construct_mapping(node, data, deep=True)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 730, in construct_mapping
github_workflow_preprocessor_1  |     return RoundTripConstructor.construct_mapping(self, node, maptyp, deep)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 1447, in construct_mapping
github_workflow_preprocessor_1  |     value = self.construct_object(value_node, deep=deep)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 144, in construct_object
github_workflow_preprocessor_1  |     data = self.construct_non_recursive_object(node)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 179, in construct_non_recursive_object
github_workflow_preprocessor_1  |     data = constructor(self, node)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 687, in construct_expression
github_workflow_preprocessor_1  |     v = exp.parse(v)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/expressions.py", line 1171, in parse
github_workflow_preprocessor_1  |     return simplify(exp[3:-2].strip())
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/expressions.py", line 1135, in simplify
github_workflow_preprocessor_1  |     o = tokens_eval(tokenizer(exp), context)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/expressions.py", line 340, in tokens_eval
github_workflow_preprocessor_1  |     assert not isinstance(t, list), t
github_workflow_preprocessor_1  | AssertionError: [Value(success), []]
modularhistory_github_workflow_preprocessor_1 exited with code 1

In case it helps, here's the workflow file that I tried to process: https://github.com/ModularHistory/modularhistory/blob/main/.github/workflows/integration.yml

(It doesn't use includes, yet.)

@mithro
Copy link
Owner

mithro commented Mar 30, 2021

It looks like you hit part of the GitHub expression language syntax I have yet to add support for.

@mithro
Copy link
Owner

mithro commented Mar 31, 2021

@iacobfred -- Hopefully 5089a36 fixes the issue with your yaml file.

@jacobjove
Copy link
Contributor Author

jacobjove commented Mar 31, 2021

It worked! And I successfully set up a pre-commit git hook (see https://pre-commit.com) to preprocess my workflow files before pushing to main. Here's what I added to my .pre-commit-config.yaml (see ):

  - repo: local
    hooks:
      - id: expand-github-workflows
        name: Expand GitHub workflows
        entry: bash -c "docker-compose up github_workflow_preprocessor"
        language: system
        always_run: true
        pass_filenames: false

Here's what I added to my docker-compose.override.yml:

  github_workflow_preprocessor:
    command: >
      python -m actions_includes ./.github/workflows-src/cleanup.yml ./.github/workflows/integration.yml
      && python -m actions_includes ./.github/workflows-src/delivery.yml ./.github/workflows/integration.yml
      && python -m actions_includes ./.github/workflows-src/integration.yml ./.github/workflows/integration.yml
      && python -m actions_includes ./.github/workflows-src/seed.yml ./.github/workflows/integration.yml
    entrypoint: ''
    image: ghcr.io/mithro/actions-includes/image:main
    user: root
    volumes:
      - .:/github/workspace
      - ./.git:/github/workspace/.git

@mithro
Copy link
Owner

mithro commented Mar 31, 2021

@iacobfred -- Could you send me some documentation updates to tell people how to do this?

@jacobjove
Copy link
Contributor Author

@mithro, sure, after I've tested that my setup is working correctly, I'll send documentation updates.

@jacobjove
Copy link
Contributor Author

jacobjove commented Apr 2, 2021

@mithro, I've noticed a small error in the documentation. It says,

As you frequently want to include local actions, actions-includes extends the {action-name} syntax to also support; /{name} - Local action under ./.github/actions/{name}.

But if I reference a local action with /setup, the tool actually looks for it in ./.github/includes/actions/setup/action.yaml (notice the includes directory), resulting in an error. I worked around this by specifying the full path, ./.github/actions/setup.

@mithro
Copy link
Owner

mithro commented Apr 2, 2021

Please send a pull request?

@jacobjove
Copy link
Contributor Author

When I tried using includes to include a composite action in my workflow (which I was not doing previously), I encountered this error:

github_workflow_preprocessor_1  | Loading yaml file /github/workspace/.github/workflows-src/integration.yml with contents md5 of 39bc3fd86d293cf1f083fde837274577
github_workflow_preprocessor_1  | get_action_data: /github/workspace/.github/workflows-src/integration.yml ./.github/actions/setup /github/workspace/.github/actions/setup
github_workflow_preprocessor_1  | Including: /github/workspace/.github/actions/setup/action.yml
github_workflow_preprocessor_1  | Loading yaml file /github/workspace/.github/actions/setup/action.yml with contents md5 of 09285d1607b6333f9c96d95b26e5c4cf
github_workflow_preprocessor_1  | Traceback (most recent call last):
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/runpy.py", line 197, in _run_module_as_main
github_workflow_preprocessor_1  |     return _run_code(code, main_globals, None,
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/runpy.py", line 87, in _run_code
github_workflow_preprocessor_1  |     exec(code, run_globals)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/__main__.py", line 25, in <module>
github_workflow_preprocessor_1  |     sys.exit(main())
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 1031, in main
github_workflow_preprocessor_1  |     out_data = expand_workflow(current_action, to_path, insert_check)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 917, in expand_workflow
github_workflow_preprocessor_1  |     data = expand_workflow_jobs(current_workflow, data)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 614, in expand_workflow_jobs
github_workflow_preprocessor_1  |     job_data = expand_job_steps(current_filepath, job_data)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 549, in expand_job_steps
github_workflow_preprocessor_1  |     steps_to_add = expand_step_includes(step_filepath, step_data)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 444, in expand_step_includes
github_workflow_preprocessor_1  |     include_yamldata = expand_input_expressions(include_yamldata, context)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 232, in expand_input_expressions
github_workflow_preprocessor_1  |     new_yaml_item[k] = expand_input_expressions(v, context)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 232, in expand_input_expressions
github_workflow_preprocessor_1  |     new_yaml_item[k] = expand_input_expressions(v, context)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 244, in expand_input_expressions
github_workflow_preprocessor_1  |     new_yaml_item.append(expand_input_expressions(yaml_item[i], context))
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 232, in expand_input_expressions
github_workflow_preprocessor_1  |     new_yaml_item[k] = expand_input_expressions(v, context)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 232, in expand_input_expressions
github_workflow_preprocessor_1  |     new_yaml_item[k] = expand_input_expressions(v, context)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 249, in expand_input_expressions
github_workflow_preprocessor_1  |     new_yaml_item = exp.eval(yaml_item, context)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/expressions.py", line 1314, in eval
github_workflow_preprocessor_1  |     new_s = RE_EXP.sub(replace_exp, s)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/expressions.py", line 1308, in replace_exp
github_workflow_preprocessor_1  |     v = simplify(e, context)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/expressions.py", line 1234, in simplify
github_workflow_preprocessor_1  |     o = tokens_eval(tokenizer(exp), context)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/expressions.py", line 354, in tokens_eval
github_workflow_preprocessor_1  |     assert not isinstance(t, list), t
github_workflow_preprocessor_1  | AssertionError: [Value(hashFiles), '**/poetry.lock']
modularhistory_github_workflow_preprocessor_1 exited with code 1

This is the step (of the composite action) that couldn't be processed:

    - name: Cache Python virtual environment
      uses: actions/[email protected]
      id: cache
      with:
        path: |
          .venv
          **/node_modules
        key: ${{ inputs.os }}-${{ hashFiles('**/poetry.lock') }}-${{ hashFiles('**/package-lock.json') }}

Here is the action file: https://github.com/ModularHistory/modularhistory/blob/MH-158/.github/actions/setup/action.yml

Here is the workflow file: https://github.com/ModularHistory/modularhistory/blob/MH-158/.github/workflows-src/integration.yml

@mithro
Copy link
Owner

mithro commented Apr 3, 2021

Caused by https://github.com/mithro/actions-includes/blob/main/actions_includes/expressions.py#L898

GitHub
Allows including an action inside another action (by preprocessing the Yaml file). This is how composite actions should have worked. - mithro/actions-includes

@mithro
Copy link
Owner

mithro commented Apr 3, 2021

@iacobfred Your issue with hashFiles should be fixed by 1813ae1 - I was successfully able to expand your integration.yml file.

@jacobjove
Copy link
Contributor Author

jacobjove commented Apr 8, 2021

Thank you @mithro. I'll create a pull request shortly to add info about using a pre-commit hook.

One more issue that I'm running into: if I try to use a || conditional operator (https://docs.github.com/en/actions/reference/context-and-expression-syntax-for-github-actions#operators) in an if statement on one of the steps in my includes action, I get this error:

github_workflow_preprocessor_1  | Loading yaml file /github/workspace/.github/actions/setup/action.yml with contents md5 of c35f7d258e95b1ec4703a3b547bd6def
github_workflow_preprocessor_1  | Traceback (most recent call last):
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/runpy.py", line 197, in _run_module_as_main
github_workflow_preprocessor_1  |     return _run_code(code, main_globals, None,
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/runpy.py", line 87, in _run_code
github_workflow_preprocessor_1  |     exec(code, run_globals)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/__main__.py", line 25, in <module>
github_workflow_preprocessor_1  |     sys.exit(main())
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 1031, in main
github_workflow_preprocessor_1  |     out_data = expand_workflow(current_action, to_path, insert_check)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 917, in expand_workflow
github_workflow_preprocessor_1  |     data = expand_workflow_jobs(current_workflow, data)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 614, in expand_workflow_jobs
github_workflow_preprocessor_1  |     job_data = expand_job_steps(current_filepath, job_data)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 549, in expand_job_steps
github_workflow_preprocessor_1  |     steps_to_add = expand_step_includes(step_filepath, step_data)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 421, in expand_step_includes
github_workflow_preprocessor_1  |     include_filepath, include_yamldata = get_action_data(current_filepath, include_step['includes'])
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 85, in get_action_data
github_workflow_preprocessor_1  |     yaml_data = yaml_load(action_filepath, data)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 817, in yaml_load
github_workflow_preprocessor_1  |     return yaml.load(yaml_data, Loader=RoundTripLoaderWithExp)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/main.py", line 1057, in load
github_workflow_preprocessor_1  |     return loader._constructor.get_single_data()
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 111, in get_single_data
github_workflow_preprocessor_1  |     return self.construct_document(node)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 121, in construct_document
github_workflow_preprocessor_1  |     for _dummy in generator:
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 1533, in construct_yaml_map
github_workflow_preprocessor_1  |     self.construct_mapping(node, data, deep=True)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 730, in construct_mapping
github_workflow_preprocessor_1  |     return RoundTripConstructor.construct_mapping(self, node, maptyp, deep)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 1447, in construct_mapping
github_workflow_preprocessor_1  |     value = self.construct_object(value_node, deep=deep)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 144, in construct_object
github_workflow_preprocessor_1  |     data = self.construct_non_recursive_object(node)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 186, in construct_non_recursive_object
github_workflow_preprocessor_1  |     for _dummy in generator:
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 1533, in construct_yaml_map
github_workflow_preprocessor_1  |     self.construct_mapping(node, data, deep=True)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 730, in construct_mapping
github_workflow_preprocessor_1  |     return RoundTripConstructor.construct_mapping(self, node, maptyp, deep)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 1447, in construct_mapping
github_workflow_preprocessor_1  |     value = self.construct_object(value_node, deep=deep)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 144, in construct_object
github_workflow_preprocessor_1  |     data = self.construct_non_recursive_object(node)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 186, in construct_non_recursive_object
github_workflow_preprocessor_1  |     for _dummy in generator:
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 1525, in construct_yaml_seq
github_workflow_preprocessor_1  |     data.extend(self.construct_rt_sequence(node, data))
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 1297, in construct_rt_sequence
github_workflow_preprocessor_1  |     ret_val.append(self.construct_object(child, deep=deep))
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 144, in construct_object
github_workflow_preprocessor_1  |     data = self.construct_non_recursive_object(node)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 186, in construct_non_recursive_object
github_workflow_preprocessor_1  |     for _dummy in generator:
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 1533, in construct_yaml_map
github_workflow_preprocessor_1  |     self.construct_mapping(node, data, deep=True)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 730, in construct_mapping
github_workflow_preprocessor_1  |     return RoundTripConstructor.construct_mapping(self, node, maptyp, deep)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 1447, in construct_mapping
github_workflow_preprocessor_1  |     value = self.construct_object(value_node, deep=deep)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 144, in construct_object
github_workflow_preprocessor_1  |     data = self.construct_non_recursive_object(node)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/ruamel/yaml/constructor.py", line 179, in construct_non_recursive_object
github_workflow_preprocessor_1  |     data = constructor(self, node)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/__init__.py", line 687, in construct_expression
github_workflow_preprocessor_1  |     v = exp.parse(v)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/expressions.py", line 1329, in parse
github_workflow_preprocessor_1  |     return simplify(exp[3:-2].strip())
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/expressions.py", line 1284, in simplify
github_workflow_preprocessor_1  |     o = tokens_eval(tokenizer(exp), context)
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/expressions.py", line 385, in tokens_eval
github_workflow_preprocessor_1  |     args.append(tokens_eval(t.pop(0), context))
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/expressions.py", line 385, in tokens_eval
github_workflow_preprocessor_1  |     args.append(tokens_eval(t.pop(0), context))
github_workflow_preprocessor_1  |   File "/usr/local/lib/python3.9/site-packages/actions_includes/expressions.py", line 373, in tokens_eval
github_workflow_preprocessor_1  |     assert not isinstance(t, list), t
github_workflow_preprocessor_1  | AssertionError: [Value(failure), []]

@mithro
Copy link
Owner

mithro commented Apr 8, 2021

Do you have an example of the string that is failing?

@mithro
Copy link
Owner

mithro commented Apr 8, 2021

It looks like there might be an interaction with || and a failure()?

@jacobjove
Copy link
Contributor Author

if: ${{ steps.venv-cache.outputs.cache-hit != 'true' || failure() }}

Perhaps I'm doing something wrong here?

@mithro
Copy link
Owner

mithro commented Apr 8, 2021

I was able to replicate this,

1356     >>> eval("${{ steps.venv-cache.outputs.cache-hit != 'true' || failure() }}", {})
UNEXPECTED EXCEPTION: AssertionError([Value(failure), []])
Traceback (most recent call last):
  File "/usr/lib/python3.9/doctest.py", line 1336, in __run
    exec(compile(example.source, filename, "single",
  File "<doctest actions_includes.expressions.eval[5]>", line 1, in <module>
  File "/ssd/github/mithro/actions-includes/actions_includes/expressions.py", line 1364, in eval
    newe = parse(s)
  File "/ssd/github/mithro/actions-includes/actions_includes/expressions.py", line 1331, in parse
    return simplify(exp[3:-2].strip())
  File "/ssd/github/mithro/actions-includes/actions_includes/expressions.py", line 1284, in simplify
    o = tokens_eval(tokenizer(exp), context)
  File "/ssd/github/mithro/actions-includes/actions_includes/expressions.py", line 385, in tokens_eval
    args.append(tokens_eval(t.pop(0), context))
  File "/ssd/github/mithro/actions-includes/actions_includes/expressions.py", line 385, in tokens_eval
    args.append(tokens_eval(t.pop(0), context))
  File "/ssd/github/mithro/actions-includes/actions_includes/expressions.py", line 373, in tokens_eval
    assert not isinstance(t, list), t
AssertionError: [Value(failure), []]

I'm pretty sure the issue is around order of operations in my expression parser. It is pretty hacky :-)

@mithro
Copy link
Owner

mithro commented Apr 8, 2021

Actually, it just seems to be the fact that I forgot to add the failure() function....

@mithro
Copy link
Owner

mithro commented Apr 8, 2021

So, I fixed the missing failure() function in 5233fab and pushed it.

It turns out there is also an order of operation bug that I need to investigate, but I don't think it will affect you in this case.

# FIXME: This is wrong!
>>> parse("${{ steps.venv-cache.outputs.cache-hit != 'true' || failure() }}")
neq(Lookup('steps', 'venv-cache', 'outputs', 'cache-hit'), or('true', failure()))

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants