Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using dependency ordering with -k #64

Open
AbdealiLoKo opened this issue Aug 29, 2022 · 5 comments
Open

Using dependency ordering with -k #64

AbdealiLoKo opened this issue Aug 29, 2022 · 5 comments
Labels
enhancement New feature or request

Comments

@AbdealiLoKo
Copy link

I was looking at pytest-dependency and came across pytest-order.
It looks pretty neat and works in the way I was hoping it would. It's amazing how this library works !

One issue I found was that when I take the example at:
https://pytest-dev.github.io/pytest-order/stable/configuration.html#order-dependencies
And run:
pytest -k test_a
It will say 1 skipped, 1 deselected

Which becomes a bit inonvenient because if I want to run a single test, I would want the "dependencies" of that test to also run.
Is there a flag I can use to "select" my test as well as it's dependencies ?

@mrbean-bremen
Copy link
Member

Great that you like the plugin!
There is no such setting, and right now I'm not sure if that is possible (e.g. if I can get to the tests before they are filtered out), but I will have a closer look at this some time later and let you know if something can be added.

@AbdealiLoKo
Copy link
Author

AbdealiLoKo commented Aug 29, 2022

@mrbean-bremen That sounds great.

I spent some time on this myself, and here is what I found by looking through the KeywordMatcher code in pytest.
Something like the below should work - we can add the markers from the child dependency to the parent dependency to make the KeywordMatcher match the parent also.
pytest seems to have extra_keyword_matches to handle this

def pytest_collection_modifyitems(session, config, items):
    from _pytest.mark import KeywordMatcher

    matchers = {}
    for item in items:
        matcher = KeywordMatcher.from_item(item)
        matchers[item] = matcher

    for item in items:
        marker = item.get_closest_marker("dependency")
        if not marker:
            continue
        depends = marker.kwargs.get('depends')
        if not depends:
            continue
        for other in items:
            if any(n in depends for n in matchers[other]._names):
                print("item", item, "depends on", other)
                other.extra_keyword_matches.update([f'd:{i}' for i in matchers[item]._names])

This does use an import from _pytest though - not sure how robust that is.
Also, I could not figure out how to handle recursive - I think if you already havet he logic to "order" the dependencies here - then it should be doable based on that ordering though

@mrbean-bremen
Copy link
Member

Thanks, that looks helpful! You can of course make a PR, if you feel like it 😄
I will see if I can put something together, but it may take a few days before I find the time.

@AbdealiLoKo
Copy link
Author

AbdealiLoKo commented Aug 30, 2022

I was having some issues with parametrized tests. And realized that pytest-order already has support for after and before which handled it a bit better
And I found the functions in pytest-order were easier to use to find "what depends on what".

Here is another config which seems to work for me with after and feels more robust:

def pytest_collection_modifyitems(items, config):
    from _pytest.mark import KeywordMatcher
    from pytest_order.sorter import Item, Sorter

    sorter = Sorter(config, items)
    sorted_items = sorter.sort_items()[::-1]

    matchers = {}
    for item in items:
        matcher = KeywordMatcher.from_item(item)
        matchers[item] = matcher

    for item in sorted_items:
        mark = item.get_closest_marker("order")
        if not mark:
            continue
        after_names = mark.kwargs.get("after")
        if not after_names:
            continue
        if isinstance(after_names, str):
            after_names = [after_names]

        for after_name in after_names:
            # We convert from pytest.Function -> pytest_order.Item to call this function
            after_items = sorter.items_from_label(after_name, Item(item), is_cls_mark=False)
            after_items = [i.item for i in after_items]

            for after_item in after_items:
                after_item.extra_keyword_matches.update([f"d:{i}" for i in matchers[item]._names])
                matchers[after_item] = KeywordMatcher.from_item(after_item)

Not sure if I will have time to create a PR as of now - but documenting in case it helps somene else

@mrbean-bremen
Copy link
Member

mrbean-bremen commented Aug 30, 2022

Thanks for that - it certainly looks promising! I'm a bit short on free time now, and also have a couple of other issues not related to pytest-order I would like to finish first, so maybe you will even be faster with a PR.
Regarding imports from _pytest: while it is not guaranteed to work in future pytest versions, the API does not change often, and this and other plugins do that all the time. So in the absence of an "official" API for the related features this is the best way to go, IMHO.

@mrbean-bremen mrbean-bremen added the enhancement New feature or request label Sep 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants