Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

remove redundant/extraneous submission_org and submission_platform fields from compliance checks before training v5.0 #395

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

matthew-frank
Copy link
Contributor

There are a number of fields in the results.txt files that were made redundant (and therefor error prone and inconsistent) when we introduced the systems/*.json files in v1.0.

This PR removes the compliance checker checks for the submission_org and submission_platform fields from the training v5.0.0 compliance checker common.yaml file. (Since a training v5.0.0 compliance checker common.yaml didn't exist yet, I copied the one from training_v4.1.0).

These two fields in the result*.txt files are redundant, and (other than their presence) are not used for anything or checked by any of the compliance or package checkers. They are difficult for many submitters to get correct in their submissions, and are often a source of confusion and/or extra work during the submission review period.

Historically, there were originally 4 fields:

  1. submission_platform which is redundant with some combination of the sys.json system_name, and number_of_nodes fields. This PR removes the compliance check for this field from training v5.0.0.
  2. submission_org which is now redundant with the sys.json submitter field. This PR removes the compliance check for this field from training v5.0.0.
  3. submission_status which was replaced by the sys.json status field, and removed from the compliance checker before the v4.1.0 round (since it used values inconsistent with the values used in the sys.json).
  4. submission_division which is now redundant with the sys.json division field. This PR does not remove the compliance check for this field from training v5.0.0 because the compliance checker uses this field to determine whether the maximal allowed initialization time is 30 minutes (closed division) or 240 minutes (open division).

@matthew-frank matthew-frank requested review from a team as code owners November 4, 2024 20:24
Copy link

github-actions bot commented Nov 4, 2024

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

@hiwotadese
Copy link
Contributor

@pgmpablo157321 can you take a look at this?

@hiwotadese
Copy link
Contributor

@pgmpablo157321 to first create v5.0 PR and add these changed on top of that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants