output of generator should be a tuple (x, y, sample_weight) or (x, y). Found: None

[RoseLii]

please help me
when i run ssd_training.ipynb，my data is 300*300,but have error
Epoch 1/120
1/2 [==============>...............] - ETA: 4s - loss: 23.7398Exception in thread Thread-6:
Traceback (most recent call last):
File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner
self.run()
File "/usr/lib/python3.5/threading.py", line 862, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.5/dist-packages/keras/engine/training.py", line 612, in data_generator_task
generator_output = next(self._generator)
File "/media/y/2T/wangxingli/TT100K/ssd_keras/data_generator/object_detection_2d_data_generator.py", line 1136, in generate
"must be homogenous in size along all axes.")
data_generator.object_detection_2d_data_generator.DegenerateBatchError: You produced an empty batch. This might be because the images in the batch vary in their size and/or number of channels. Note that after all transformations (if any were given) have been applied to all images in the batch, all images must be homogenous in size along all axes.

Traceback (most recent call last):
File "ssd300_training.py", line 457, in
initial_epoch=initial_epoch)
File "/usr/local/lib/python3.5/dist-packages/keras/legacy/interfaces.py", line 88, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/keras/engine/training.py", line 1924, in fit_generator
pickle_safe=pickle_safe)
File "/usr/local/lib/python3.5/dist-packages/keras/legacy/interfaces.py", line 88, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/keras/engine/training.py", line 2010, in evaluate_generator
str(generator_output))
ValueError: output of generator should be a tuple (x, y, sample_weight) or (x, y). Found: None
I check my data sets, no size is not consistent, and I in the other size of the data set can be, I've been looking around for a day, I have not found any problem, please help me

[pierluigiferrari]

This is the kind of issue that you can debug yourself relatively easily with a few simple print statements inside DataGenerator.generate(). Work your way backwards from the point where the DegenerateBatchError is being raised and you will eventually see where it breaks. First, print out the shape of all images in the batch just before the line

batch_X = np.array(batch_X)

You'll find out that either of the following two cases occur.
Either

1. the images in the list batch_X do not agree in all dimensions, i.e. at least one of height, width and number of channels is not the same for all images in the batch, in which case converting the list into an array will result in an empty array,
   or
2. the list batch_X was already empty before the conversion into an array, in which case you go back further step by step until you find out where you're losing the images.

And so on.

There is no magic going on here, you simply examine what the content of batch_X looks like at the various points in the code from the exception backwards and after no more than 10 minutes you will probably know what the problem is if you go about it systematically.

[RoseLii]

@pierluigiferrari Thank you very much for your suggestion. There is indeed an empty array in the print.and I will print out the image name to see which one it is .but after changing batch_sise, every time something goes wrong, the picture is different, so I don't know how to solve it. Please help me

[pierluigiferrari]

It doesn't matter what the concrete pictures in a given batch are or that they change in every new batch. You still simply proceed with the debugging process I laid out above.

[RoseLii]

Please help me. There are some images in my test set that have no target, so there is no label. The following error is now detected:
File "ssd300_evaluation.py", line 192, in
verbose=True)
File "/media/y/2T/wangxingli/TT100K/ssd_keras/eval_utils/average_precision_evaluator.py", line 201, in call
ret=False)
File "/media/y/2T/wangxingli/TT100K/ssd_keras/eval_utils/average_precision_evaluator.py", line 376, in predict_on_dataset
batch_X, batch_image_ids, batch_eval_neutral, batch_inverse_transforms, batch_orig_labels = next(generator)
File "/media/y/2T/wangxingli/TT100K/ssd_keras/data_generator/object_detection_2d_data_generator.py", line 1071, in generate
batch_X[i], batch_y[i], inverse_transform = transform(batch_X[i], batch_y[i], return_inverter=True)
File "/media/y/2T/wangxingli/TT100K/ssd_keras/data_generator/object_detection_2d_geometric_ops.py", line 88, in call
labels[:, [ymin, ymax]] = np.round(labels[:, [ymin, ymax]] * (self.out_height / img_height), decimals=0)
IndexError: too many indices for array
I know that because I read the file as a whole, rather than one by one, I made an error in the unlabeled file. How should I change the code? Trouble you

[pierluigiferrari]

What you posted in your last post might actually be a bug. The data generator works fine if all images have ground truth or if none of the images have ground truth, but if only some images don't have ground truth, it treats them as if they should have ground truth. I will fix this as soon as I find the time.

In general, please mind the following things:

1. If you post code or a stack trace, please format it as code. This makes it a lot easier to read. Reading through the traceback above is a pain in the ass because it isn't formatted properly.
2. Your last post is a new issue. It has nothing to do with your original issue above. Please always open a new GitHub issue in such cases, don't put multiple distinct bugs/issues/problems in the same GitHub issue.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[jetjodh]

HI @pierluigiferrari , have you fixed this bug yet?

[pierluigiferrari]

@jetjodh No, I'm busy with other stuff at the moment, but in #143 I proposed a possible solution that you could try out locally until I find the time to test and push a fix.