Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error when attempting to optimize stable-diffusion with directML #1267

Open
fragfeaster777 opened this issue Jul 28, 2024 · 6 comments
Open
Labels
DirectML DirectML

Comments

@fragfeaster777
Copy link

When attempting: python stable_diffusion.py --optimize, I get a "TypeError: z_(): incompatible function arguments" error for "Optimizing text_encoder". Note that "Optimizing vae_encoder", "Optimizing vae_decoder" and "Optimizing unet" worked.

Log under "Optimizing text_encoder":

[DEBUG] [olive_evaluator.py:1153:validate_metrics] No priority is specified, but only one sub type metric is specified. Use rank 1 for single for this metric.
[INFO] [run.py:138:run_engine] Running workflow default_workflow
[INFO] [engine.py:986:save_olive_config] Saved Olive config to cache\default_workflow\olive_config.json
[DEBUG] [run.py:179:run_engine] Registering pass OnnxConversion
[DEBUG] [run.py:179:run_engine] Registering pass OrtTransformersOptimization
[DEBUG] [accelerator_creator.py:130:_fill_accelerators] The accelerator device and execution providers are specified, skipping deduce.
[DEBUG] [accelerator_creator.py:169:_check_execution_providers] Supported execution providers for device gpu: ['DmlExecutionProvider', 'CPUExecutionProvider']
[DEBUG] [accelerator_creator.py:199:create_accelerators] Initial accelerators and execution providers: {'gpu': ['DmlExecutionProvider']}
[INFO] [accelerator_creator.py:224:create_accelerators] Running workflow on accelerator specs: gpu-dml
[DEBUG] [run.py:235:run_engine] Pass OnnxConversion already registered
[DEBUG] [run.py:235:run_engine] Pass OpenVINOConversion already registered
[DEBUG] [run.py:235:run_engine] Pass OrtTransformersOptimization already registered
[DEBUG] [run.py:235:run_engine] Pass OrtTransformersOptimization already registered
[INFO] [engine.py:109:initialize] Using cache directory: cache\default_workflow
[INFO] [engine.py:265:run] Running Olive on accelerator: gpu-dml
[INFO] [engine.py:1085:_create_system] Creating target system ...
[DEBUG] [engine.py:1081:create_system] create native OliveSystem SystemType.Local
[INFO] [engine.py:1088:_create_system] Target system created in 0.000000 seconds
[INFO] [engine.py:1097:_create_system] Creating host system ...
[DEBUG] [engine.py:1081:create_system] create native OliveSystem SystemType.Local
[INFO] [engine.py:1100:_create_system] Host system created in 0.000000 seconds
[DEBUG] [engine.py:711:_cache_model] Cached model bebe0e3c to cache\default_workflow\models\bebe0e3c.json
[DEBUG] [engine.py:338:run_accelerator] Running Olive in no-search mode ...
[DEBUG] [engine.py:430:run_no_search] Running ['convert', 'optimize'] with no search ...
[INFO] [engine.py:867:_run_pass] Running pass convert:OnnxConversion
[DEBUG] [resource_path.py:156:create_resource_path] Resource path runwayml/stable-diffusion-v1-5 is inferred to be of type string_name.
[DEBUG] [resource_path.py:156:create_resource_path] Resource path user_script.py is inferred to be of type file.
[DEBUG] [resource_path.py:156:create_resource_path] Resource path runwayml/stable-diffusion-v1-5 is inferred to be of type string_name.
[DEBUG] [resource_path.py:156:create_resource_path] Resource path user_script.py is inferred to be of type file.
[DEBUG] [resource_path.py:156:create_resource_path] Resource path C:\Users\ih\sd-test\converter\olive\examples\stable_diffusion\user_script.py is inferred to be of type file.
[DEBUG] [dummy_inputs.py:45:get_dummy_inputs] Using dummy_inputs_func to get dummy inputs
[DEBUG] [conversion.py:234:_export_pytorch_model] Converting model on device cpu with dtype None.
Traceback (most recent call last):
File "C:\Users\ih\sd-test\converter\olive\examples\stable_diffusion\stable_diffusion.py", line 433, in
main()
File "C:\Users\ih\sd-test\converter\olive\examples\stable_diffusion\stable_diffusion.py", line 370, in main
optimize(common_args.model_id, common_args.provider, unoptimized_model_dir, optimized_model_dir)
File "C:\Users\ih\sd-test\converter\olive\examples\stable_diffusion\stable_diffusion.py", line 244, in optimize
run_res = olive_run(olive_config)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\olive\workflows\run\run.py", line 297, in run
return run_engine(package_config, run_config, data_root)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\olive\workflows\run\run.py", line 261, in run_engine
engine.run(
File "C:\Users\ih.conda\envs\converter\lib\site-packages\olive\engine\engine.py", line 267, in run
run_result = self.run_accelerator(
File "C:\Users\ih.conda\envs\converter\lib\site-packages\olive\engine\engine.py", line 339, in run_accelerator
output_footprint = self.run_no_search(
File "C:\Users\ih.conda\envs\converter\lib\site-packages\olive\engine\engine.py", line 431, in run_no_search
should_prune, signal, model_ids = self._run_passes(
File "C:\Users\ih.conda\envs\converter\lib\site-packages\olive\engine\engine.py", line 829, in _run_passes
model_config, model_id = self._run_pass(
File "C:\Users\ih.conda\envs\converter\lib\site-packages\olive\engine\engine.py", line 937, in _run_pass
output_model_config = host.run_pass(p, input_model_config, data_root, output_model_path, pass_search_point)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\olive\systems\local.py", line 32, in run_pass
output_model = the_pass.run(model, data_root, output_model_path, point)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\olive\passes\olive_pass.py", line 224, in run
output_model = self._run_for_config(model, data_root, config, output_model_path)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\olive\passes\onnx\conversion.py", line 132, in _run_for_config
output_model = self._run_for_config_internal(model, data_root, config, output_model_path)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\olive\passes\onnx\conversion.py", line 182, in _run_for_config_internal
return self._convert_model_on_device(model, data_root, config, output_model_path, device, torch_dtype)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\olive\passes\onnx\conversion.py", line 439, in _convert_model_on_device
converted_onnx_model = OnnxConversion._export_pytorch_model(
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\utils_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\olive\passes\onnx\conversion.py", line 285, in _export_pytorch_model
torch.onnx.export(
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx\utils.py", line 551, in export
_export(
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx\utils.py", line 1648, in _export
graph, params_dict, torch_out = _model_to_graph(
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx\utils.py", line 1174, in _model_to_graph
graph = _optimize_graph(
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx\utils.py", line 714, in _optimize_graph
graph = _C._jit_pass_onnx(graph, operator_export_type)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx\utils.py", line 1997, in _run_symbolic_function
return symbolic_fn(graph_context, *inputs, **attrs)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx\symbolic_helper.py", line 292, in wrapper
return fn(g, *args, **kwargs)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx\symbolic_opset14.py", line 177, in scaled_dot_product_attention
query_scaled = g.op("Mul", query, g.op("Sqrt", scale))
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx_internal\jit_utils.py", line 93, in op
return _add_op(self, opname, *raw_args, outputs=outputs, **kwargs)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx_internal\jit_utils.py", line 244, in _add_op
inputs = [_const_if_tensor(graph_context, arg) for arg in args]
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx_internal\jit_utils.py", line 244, in
inputs = [_const_if_tensor(graph_context, arg) for arg in args]
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx_internal\jit_utils.py", line 276, in _const_if_tensor
return _add_op(graph_context, "onnx::Constant", value_z=arg)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx_internal\jit_utils.py", line 252, in _add_op
node = _create_node(
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx_internal\jit_utils.py", line 312, in _create_node
add_attribute(node, key, value, aten=aten)
File "C:\Users\ih.conda\envs\converter\lib\site-packages\torch\onnx_internal\jit_utils.py", line 363, in add_attribute
return getattr(node, f"{kind}
")(name, value)
TypeError: z
(): incompatible function arguments. The following argument types are supported:
1. (self: torch._C.Node, arg0: str, arg1: torch.Tensor) -> torch._C.Node

Invoked with: %340 : Tensor = onnx::Constant(), scope: transformers.models.clip.modeling_clip.CLIPTextModel::/transformers.models.clip.modeling_clip.CLIPTextTransformer::text_model/transformers.models.clip.modeling_clip.CLIPEncoder::encoder/transformers.models.clip.modeling_clip.CLIPEncoderLayer::layers.0/transformers.models.clip.modeling_clip.CLIPSdpaAttention::self_attn
, 'value', 0.125
(Occurred when translating scaled_dot_product_attention).

Other information

  • OS: Windows 10
  • Olive version: Tried with 0.6.2 and 0.4.0
  • onnxruntime-directml: 1.18.0
@xiaoyu-work
Copy link
Contributor

@PatriceVignola can you please take a look?

@xiaoyu-work xiaoyu-work added the DirectML DirectML label Aug 2, 2024
@DominicSYHsieh
Copy link

I have some almost the same issues, here's the error log :

Optimizing text_encoder
[2024-08-08 02:36:04,730] [INFO] [run.py:138:run_engine] Running workflow default_workflow
[2024-08-08 02:36:04,733] [INFO] [engine.py:986:save_olive_config] Saved Olive config to cache\default_workflow\olive_config.json
[2024-08-08 02:36:04,733] [INFO] [accelerator_creator.py:224:create_accelerators] Running workflow on accelerator specs: gpu-dml
[2024-08-08 02:36:04,733] [INFO] [engine.py:109:initialize] Using cache directory: cache\default_workflow
[2024-08-08 02:36:04,735] [INFO] [engine.py:265:run] Running Olive on accelerator: gpu-dml
[2024-08-08 02:36:04,735] [INFO] [engine.py:1085:_create_system] Creating target system ...
[2024-08-08 02:36:04,735] [INFO] [engine.py:1088:_create_system] Target system created in 0.000000 seconds
[2024-08-08 02:36:04,735] [INFO] [engine.py:1097:_create_system] Creating host system ...
[2024-08-08 02:36:04,735] [INFO] [engine.py:1100:_create_system] Host system created in 0.000000 seconds
[2024-08-08 02:36:04,773] [INFO] [engine.py:867:_run_pass] Running pass convert:OnnxConversion
Traceback (most recent call last):
File "C:\Users\Acer\Desktop\Olive\examples\directml\stable_diffusion_xl\stable_diffusion_xl.py", line 635, in
main()
File "C:\Users\Acer\Desktop\Olive\examples\directml\stable_diffusion_xl\stable_diffusion_xl.py", line 601, in main
optimize(
File "C:\Users\Acer\Desktop\Olive\examples\directml\stable_diffusion_xl\stable_diffusion_xl.py", line 369, in optimize
olive_run(olive_config)
File "C:\Users\Acer\Desktop\Olive\olive\workflows\run\run.py", line 297, in run
return run_engine(package_config, run_config, data_root)
File "C:\Users\Acer\Desktop\Olive\olive\workflows\run\run.py", line 261, in run_engine
engine.run(
File "C:\Users\Acer\Desktop\Olive\olive\engine\engine.py", line 267, in run
run_result = self.run_accelerator(
File "C:\Users\Acer\Desktop\Olive\olive\engine\engine.py", line 339, in run_accelerator
output_footprint = self.run_no_search(
File "C:\Users\Acer\Desktop\Olive\olive\engine\engine.py", line 431, in run_no_search
should_prune, signal, model_ids = self._run_passes(
File "C:\Users\Acer\Desktop\Olive\olive\engine\engine.py", line 829, in _run_passes
model_config, model_id = self._run_pass(
File "C:\Users\Acer\Desktop\Olive\olive\engine\engine.py", line 937, in _run_pass
output_model_config = host.run_pass(p, input_model_config, data_root, output_model_path, pass_search_point)
File "C:\Users\Acer\Desktop\Olive\olive\systems\local.py", line 32, in run_pass
output_model = the_pass.run(model, data_root, output_model_path, point)
File "C:\Users\Acer\Desktop\Olive\olive\passes\olive_pass.py", line 224, in run
output_model = self._run_for_config(model, data_root, config, output_model_path)
File "C:\Users\Acer\Desktop\Olive\olive\passes\onnx\conversion.py", line 132, in _run_for_config
output_model = self._run_for_config_internal(model, data_root, config, output_model_path)
File "C:\Users\Acer\Desktop\Olive\olive\passes\onnx\conversion.py", line 182, in _run_for_config_internal
return self._convert_model_on_device(model, data_root, config, output_model_path, device, torch_dtype)
File "C:\Users\Acer\Desktop\Olive\olive\passes\onnx\conversion.py", line 439, in _convert_model_on_device
converted_onnx_model = OnnxConversion._export_pytorch_model(
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\utils_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
File "C:\Users\Acer\Desktop\Olive\olive\passes\onnx\conversion.py", line 285, in _export_pytorch_model
torch.onnx.export(
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx\utils.py", line 551, in export
_export(
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx\utils.py", line 1648, in _export
graph, params_dict, torch_out = _model_to_graph(
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx\utils.py", line 1174, in _model_to_graph
graph = _optimize_graph(
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx\utils.py", line 714, in _optimize_graph
graph = _C._jit_pass_onnx(graph, operator_export_type)
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx\utils.py", line 1997, in _run_symbolic_function
return symbolic_fn(graph_context, *inputs, **attrs)
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx\symbolic_helper.py", line 292, in wrapper
return fn(g, *args, **kwargs)
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx\symbolic_opset14.py", line 177, in scaled_dot_product_attention
query_scaled = g.op("Mul", query, g.op("Sqrt", scale))
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx_internal\jit_utils.py", line 93, in op
return _add_op(self, opname, *raw_args, outputs=outputs, **kwargs)
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx_internal\jit_utils.py", line 244, in _add_op
inputs = [_const_if_tensor(graph_context, arg) for arg in args]
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx_internal\jit_utils.py", line 244, in
inputs = [_const_if_tensor(graph_context, arg) for arg in args]
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx_internal\jit_utils.py", line 276, in _const_if_tensor
return _add_op(graph_context, "onnx::Constant", value_z=arg)
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx_internal\jit_utils.py", line 252, in _add_op
node = _create_node(
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx_internal\jit_utils.py", line 312, in _create_node
add_attribute(node, key, value, aten=aten)
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx_internal\jit_utils.py", line 363, in add_attribute
return getattr(node, f"{kind}
")(name, value)
TypeError: z
(): incompatible function arguments. The following argument types are supported:
1. (self: torch._C.Node, arg0: str, arg1: torch.Tensor) -> torch._C.Node

Invoked with: %341 : Tensor = onnx::Constant(), scope: transformers.models.clip.modeling_clip.CLIPTextModel::/transformers.models.clip.modeling_clip.CLIPTextTransformer::text_model/transformers.models.clip.modeling_clip.CLIPEncoder::encoder/transformers.models.clip.modeling_clip.CLIPEncoderLayer::layers.0/transformers.models.clip.modeling_clip.CLIPSdpaAttention::self_attn
, 'value', 0.125
(Occurred when translating scaled_dot_product_attention).

as well as the text_encoder_2, error log:

Optimizing text_encoder_2
[2024-08-08 02:24:50,086] [INFO] [run.py:138:run_engine] Running workflow default_workflow
[2024-08-08 02:24:50,089] [INFO] [engine.py:986:save_olive_config] Saved Olive config to cache\default_workflow\olive_config.json
[2024-08-08 02:24:50,090] [INFO] [accelerator_creator.py:224:create_accelerators] Running workflow on accelerator specs: gpu-dml
[2024-08-08 02:24:50,090] [INFO] [engine.py:109:initialize] Using cache directory: cache\default_workflow
[2024-08-08 02:24:50,090] [INFO] [engine.py:265:run] Running Olive on accelerator: gpu-dml
[2024-08-08 02:24:50,090] [INFO] [engine.py:1085:_create_system] Creating target system ...
[2024-08-08 02:24:50,090] [INFO] [engine.py:1088:_create_system] Target system created in 0.000000 seconds
[2024-08-08 02:24:50,090] [INFO] [engine.py:1097:_create_system] Creating host system ...
[2024-08-08 02:24:50,090] [INFO] [engine.py:1100:_create_system] Host system created in 0.000000 seconds
[2024-08-08 02:24:50,136] [INFO] [engine.py:867:_run_pass] Running pass convert:OnnxConversion
Traceback (most recent call last):
File "C:\Users\Acer\Desktop\Olive\examples\directml\stable_diffusion_xl\stable_diffusion_xl.py", line 635, in
main()
File "C:\Users\Acer\Desktop\Olive\examples\directml\stable_diffusion_xl\stable_diffusion_xl.py", line 601, in main
optimize(
File "C:\Users\Acer\Desktop\Olive\examples\directml\stable_diffusion_xl\stable_diffusion_xl.py", line 369, in optimize
olive_run(olive_config)
File "C:\Users\Acer\Desktop\Olive\olive\workflows\run\run.py", line 297, in run
return run_engine(package_config, run_config, data_root)
File "C:\Users\Acer\Desktop\Olive\olive\workflows\run\run.py", line 261, in run_engine
engine.run(
File "C:\Users\Acer\Desktop\Olive\olive\engine\engine.py", line 267, in run
run_result = self.run_accelerator(
File "C:\Users\Acer\Desktop\Olive\olive\engine\engine.py", line 339, in run_accelerator
output_footprint = self.run_no_search(
File "C:\Users\Acer\Desktop\Olive\olive\engine\engine.py", line 431, in run_no_search
should_prune, signal, model_ids = self._run_passes(
File "C:\Users\Acer\Desktop\Olive\olive\engine\engine.py", line 829, in _run_passes
model_config, model_id = self._run_pass(
File "C:\Users\Acer\Desktop\Olive\olive\engine\engine.py", line 937, in _run_pass
output_model_config = host.run_pass(p, input_model_config, data_root, output_model_path, pass_search_point)
File "C:\Users\Acer\Desktop\Olive\olive\systems\local.py", line 32, in run_pass
output_model = the_pass.run(model, data_root, output_model_path, point)
File "C:\Users\Acer\Desktop\Olive\olive\passes\olive_pass.py", line 224, in run
output_model = self._run_for_config(model, data_root, config, output_model_path)
File "C:\Users\Acer\Desktop\Olive\olive\passes\onnx\conversion.py", line 132, in _run_for_config
output_model = self._run_for_config_internal(model, data_root, config, output_model_path)
File "C:\Users\Acer\Desktop\Olive\olive\passes\onnx\conversion.py", line 182, in _run_for_config_internal
return self._convert_model_on_device(model, data_root, config, output_model_path, device, torch_dtype)
File "C:\Users\Acer\Desktop\Olive\olive\passes\onnx\conversion.py", line 439, in _convert_model_on_device
converted_onnx_model = OnnxConversion._export_pytorch_model(
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\utils_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
File "C:\Users\Acer\Desktop\Olive\olive\passes\onnx\conversion.py", line 285, in _export_pytorch_model
torch.onnx.export(
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx\utils.py", line 551, in export
_export(
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx\utils.py", line 1648, in _export
graph, params_dict, torch_out = _model_to_graph(
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx\utils.py", line 1174, in _model_to_graph
graph = _optimize_graph(
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx\utils.py", line 714, in _optimize_graph
graph = _C._jit_pass_onnx(graph, operator_export_type)
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx\utils.py", line 1997, in _run_symbolic_function
return symbolic_fn(graph_context, *inputs, **attrs)
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx\symbolic_helper.py", line 292, in wrapper
return fn(g, *args, **kwargs)
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx\symbolic_opset14.py", line 177, in scaled_dot_product_attention
query_scaled = g.op("Mul", query, g.op("Sqrt", scale))
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx_internal\jit_utils.py", line 93, in op
return _add_op(self, opname, *raw_args, outputs=outputs, **kwargs)
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx_internal\jit_utils.py", line 244, in _add_op
inputs = [_const_if_tensor(graph_context, arg) for arg in args]
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx_internal\jit_utils.py", line 244, in
inputs = [_const_if_tensor(graph_context, arg) for arg in args]
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx_internal\jit_utils.py", line 276, in _const_if_tensor
return _add_op(graph_context, "onnx::Constant", value_z=arg)
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx_internal\jit_utils.py", line 252, in _add_op
node = _create_node(
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx_internal\jit_utils.py", line 312, in _create_node
add_attribute(node, key, value, aten=aten)
File "C:\Users\Acer\anaconda3\envs\olive_sd\lib\site-packages\torch\onnx_internal\jit_utils.py", line 363, in add_attribute
return getattr(node, f"{kind}
")(name, value)
TypeError: z
(): incompatible function arguments. The following argument types are supported:
1. (self: torch._C.Node, arg0: str, arg1: torch.Tensor) -> torch._C.Node

Invoked with: %662 : Tensor = onnx::Constant(), scope: transformers.models.clip.modeling_clip.CLIPTextModelWithProjection::/transformers.models.clip.modeling_clip.CLIPTextTransformer::text_model/transformers.models.clip.modeling_clip.CLIPEncoder::encoder/transformers.models.clip.modeling_clip.CLIPEncoderLayer::layers.0/transformers.models.clip.modeling_clip.CLIPSdpaAttention::self_attn
, 'value', 0.125
(Occurred when translating scaled_dot_product_attention).

System information :
OS: Windows 11
Olive : 0.6.2
onnxruntime-directml : tried with 1.18.0 / 1.18.1

@Jay19751103
Copy link

Use following can optimize the model, it needs to change json
accelerate==0.33.0
aiohappyeyeballs==2.3.4
aiohttp==3.10.1
aiosignal==1.3.1
alembic==1.13.2
annotated-types==0.7.0
async-timeout==4.0.3
attrs==24.2.0
certifi==2024.7.4
charset-normalizer==3.3.2
colorama==0.4.6
coloredlogs==15.0.1
colorlog==6.8.2
config==0.5.1
datasets==2.20.0
diffusers==0.29.2
dill==0.3.8
filelock==3.15.4
flatbuffers==24.3.25
frozenlist==1.4.1
fsspec==2024.5.0
greenlet==3.0.3
huggingface-hub==0.24.5
humanfriendly==10.0
idna==3.7
importlib_metadata==8.2.0
invisible-watermark==0.2.0
Jinja2==3.1.4
lightning-utilities==0.11.6
Mako==1.3.5
MarkupSafe==2.1.5
mpmath==1.3.0
multidict==6.0.5
multiprocess==0.70.16
networkx==3.3
numpy==1.26.4
olive-ai==0.6.2
onnx==1.16.2
onnx-tool==0.9.0
onnx_opcounter==0.0.3
onnxruntime==1.18.0
onnxruntime-directml==1.18.0
opencv-python==4.10.0.84
optimum==1.21.3
optuna==3.6.1
packaging==24.1
pandas==2.2.2
pillow==10.4.0
protobuf==3.20.3
psutil==6.0.0
pyarrow==17.0.0
pyarrow-hotfix==0.6
pydantic==2.7.0
pydantic_core==2.18.1
pyreadline3==3.4.1
python-dateutil==2.9.0.post0
pytz==2024.1
PyWavelets==1.6.0
PyYAML==6.0.1
regex==2024.7.24
requests==2.32.3
safetensors==0.4.4
sentencepiece==0.2.0
six==1.16.0
SQLAlchemy==2.0.32
sympy==1.13.1
tabulate==0.9.0
tokenizers==0.19.1
torch==2.4.0
torchmetrics==1.4.1
tqdm==4.66.5
transformers==4.42.4
typing_extensions==4.12.2
tzdata==2024.1
urllib3==2.2.2
xxhash==3.4.1
yarl==1.9.4
zipp==3.19.2

OLIVE modifications
0001-Fix-optimization-sdxl-issues.patch

Note that weights.pb does not copied into optimized unet folder
find in from cache or footprint directory , the size close 5GB and copy it to the folder manually, then you can run it locally.

@Jay19751103
Copy link

Key for this issue is downgrade transformer to transformers==4.42.4

@saddam213
Copy link

I am also stuck on this issue

TypeError: z_(): incompatible function arguments. The following argument types are supported:
    1. (self: torch._C.Node, arg0: str, arg1: torch.Tensor) -> torch._C.Node

Invoked with: %329 : Tensor = onnx::Constant(), scope: transformers.models.clip.modeling_clip.CLIPTextModel::/transformers.models.clip.modeling_clip.CLIPTextTransformer::text_model/transformers.models.clip.modeling_clip.CLIPEncoder::encoder/transformers.models.clip.modeling_clip.CLIPEncoderLayer::layers.0/transformers.models.clip.modeling_clip.CLIPSdpaAttention::self_attn
, 'value', 0.125
(Occurred when translating scaled_dot_product_attention).

This has broken all stable-diffusion converter scripts

@DominicSYHsieh
Copy link

For this issue, downgrade transformers to 4.42.4 is work for me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
DirectML DirectML
Projects
None yet
Development

No branches or pull requests

5 participants