-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support uv compiled requirements files #10040
base: main
Are you sure you want to change the base?
Changes from 9 commits
3a82e1d
5a9eb3f
4d67cb2
c4fba47
cd7beaf
8338b77
afa5d74
2edf6b2
1c081bf
7965aeb
c9d1c92
cbc1e12
55a1a26
88fb54b
39cc41e
fc2e796
ef18227
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
|
@@ -96,13 +96,14 @@ def compile_new_requirement_files | |||||
def compile_file(filename) | ||||||
# Shell out to pip-compile, generate a new set of requirements. | ||||||
# This is slow, as pip-compile needs to do installs. | ||||||
options = pip_compile_options(filename) | ||||||
|
||||||
options, command = pip_compile_options(filename) | ||||||
options_fingerprint = pip_compile_options_fingerprint(options) | ||||||
|
||||||
name_part = "pyenv exec pip-compile " \ | ||||||
name_part = "#{command} " \ | ||||||
"#{options} -P " \ | ||||||
"#{dependency.name}" | ||||||
fingerprint_name_part = "pyenv exec pip-compile " \ | ||||||
fingerprint_name_part = "#{command} " \ | ||||||
"#{options_fingerprint} -P " \ | ||||||
"<dependency_name>" | ||||||
|
||||||
|
@@ -453,10 +454,16 @@ def pip_compile_options(filename) | |||||
options += pip_compile_index_options | ||||||
|
||||||
if (requirements_file = compiled_file_for_filename(filename)) | ||||||
options += pip_compile_options_from_compiled_file(requirements_file) | ||||||
if requirements_file.content.include?("uv pip compile") | ||||||
options += uv_pip_compile_options_from_compiled_file(requirements_file) | ||||||
command = "pyenv exec uv pip compile" | ||||||
else | ||||||
options += pip_compile_options_from_compiled_file(requirements_file) | ||||||
command = "pyenv exec pip-compile" | ||||||
end | ||||||
end | ||||||
|
||||||
options.join(" ") | ||||||
[options.join(" "), command] | ||||||
end | ||||||
|
||||||
def pip_compile_options_from_compiled_file(requirements_file) | ||||||
|
@@ -483,6 +490,32 @@ def pip_compile_options_from_compiled_file(requirements_file) | |||||
options | ||||||
end | ||||||
|
||||||
def uv_pip_compile_options_from_compiled_file(requirements_file) | ||||||
options = ["--output-file=#{requirements_file.name}"] | ||||||
|
||||||
options << "--no-emit-index-url" unless requirements_file.content.include?("index-url http") | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. pip-tools defaults to uv defaults to
Suggested change
|
||||||
|
||||||
options << "--generate-hashes" if requirements_file.content.include?("--hash=sha") | ||||||
|
||||||
options << "--no-annotate" unless requirements_file.content.include?("# via ") | ||||||
|
||||||
options << "--no-header" | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This line slightly confuses me, as wouldn't this remove the only marker that makes it possible to identify uv compiled locks? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is it possible to specifically request uv instead of pip-tools when no header is present? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Dependabot currently doesn't allow specifying what tooling to use for compiled dependencies to my knowledge, it just has been Maybe in the future, configuration could be added for it, but I don't think this is the case, so keeping the header is most probably something that is wanted. Otherwise, as the current code stands, I don't think it will even be able to pickup that it's a dependency compilation file (even for
avilaton marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
|
||||||
options << "--pre" if requirements_file.content.include?("--pre") | ||||||
|
||||||
options << "--no-strip-extras" if requirements_file.content.include?("--no-strip-extras") | ||||||
|
||||||
if requirements_file.content.include?("--no-binary") || requirements_file.content.include?("--only-binary") | ||||||
options << "--emit-build-options" | ||||||
end | ||||||
|
||||||
if (resolver = RESOLVER_REGEX.match(requirements_file.content)) | ||||||
options << "--resolver=#{resolver}" | ||||||
end | ||||||
Comment on lines
+510
to
+512
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't think there's any reason to pass this to uv, since it's just there for better UX |
||||||
|
||||||
options | ||||||
avilaton marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
end | ||||||
|
||||||
def pip_compile_index_options | ||||||
credentials | ||||||
.select { |cred| cred["type"] == "python_index" } | ||||||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -91,12 +91,12 @@ def fetch_latest_resolvable_version_string(requirement:) | |
def compile_file(filename) | ||
# Shell out to pip-compile. | ||
# This is slow, as pip-compile needs to do installs. | ||
options = pip_compile_options(filename) | ||
options, command = pip_compile_options(filename) | ||
options_fingerprint = pip_compile_options_fingerprint(options) | ||
|
||
run_pip_compile_command( | ||
"pyenv exec pip-compile -v #{options} -P #{dependency.name} #{filename}", | ||
fingerprint: "pyenv exec pip-compile -v #{options_fingerprint} -P <dependency_name> <filename>" | ||
"#{command} -v #{options} -P #{dependency.name} #{filename}", | ||
fingerprint: "#{command} -v #{options_fingerprint} -P <dependency_name> <filename>" | ||
) | ||
|
||
return true if dependency.top_level? | ||
|
@@ -110,8 +110,8 @@ def compile_file(filename) | |
# update_not_possible. | ||
write_original_manifest_files | ||
run_pip_compile_command( | ||
"pyenv exec pip-compile #{options} #{filename}", | ||
fingerprint: "pyenv exec pip-compile #{options_fingerprint} <filename>" | ||
"#{command} #{options} #{filename}", | ||
fingerprint: "#{command} #{options_fingerprint} <filename>" | ||
) | ||
|
||
true | ||
|
@@ -201,12 +201,12 @@ def check_original_requirements_resolvable | |
write_temporary_dependency_files(update_requirement: false) | ||
|
||
filenames_to_compile.each do |filename| | ||
options = pip_compile_options(filename) | ||
options, command = pip_compile_options(filename) | ||
options_fingerprint = pip_compile_options_fingerprint(options) | ||
|
||
run_pip_compile_command( | ||
"pyenv exec pip-compile #{options} #{filename}", | ||
fingerprint: "pyenv exec pip-compile #{options_fingerprint} <filename>" | ||
"#{command} #{options} #{filename}", | ||
fingerprint: "#{command} #{options_fingerprint} <filename>" | ||
) | ||
end | ||
|
||
|
@@ -251,7 +251,13 @@ def pip_compile_options(filename) | |
options << "--output-file=#{requirements_file.name}" | ||
end | ||
|
||
options.join(" ") | ||
command = "pyenv exec pip-compile" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Nitpick, but if else to set the command seems cleaner (similar to the other place in the code where this check is done) |
||
if (requirements_file = compiled_file_for_filename(filename)) && | ||
requirements_file.content.include?("autogenerated by uv") | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This checks a different string than the other function to generate flags from an existing requirements file. Would it be worth extracting the text we search for and making them both use the same?
avilaton marked this conversation as resolved.
Show resolved
Hide resolved
|
||
command = "pyenv exec uv pip compile" | ||
end | ||
|
||
[options.join(" "), command] | ||
end | ||
|
||
def pip_compile_index_options | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This definitely needs a version bump