Eval saving failing for custom tool calling #547
Description
When running:
uv run vf-eval notion-bench -s
It throws the following issue:
Traceback (most recent call last):
File "/prime-environments/.venv/bin/vf-eval", line 10, in <module>
sys.exit(main())
^^^^^^
File "/prime-environments/.venv/lib/python3.12/site-packages/verifiers/scripts/eval.py", line 333, in main
eval_environment(
File "/prime-environments/.venv/lib/python3.12/site-packages/verifiers/scripts/eval.py", line 177, in eval_environment
dataset = Dataset.from_dict(data_dict)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/prime-environments/.venv/lib/python3.12/site-packages/datasets/arrow_dataset.py", line 1009, in from_dict
pa_table = InMemoryTable.from_pydict(mapping=mapping)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/prime-environments/.venv/lib/python3.12/site-packages/datasets/table.py", line 757, in from_pydict
return cls(pa.Table.from_pydict(*args, **kwargs))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "pyarrow/table.pxi", line 1985, in pyarrow.lib._Tabular.from_pydict
File "pyarrow/table.pxi", line 6400, in pyarrow.lib._from_pydict
File "pyarrow/array.pxi", line 405, in pyarrow.lib.asarray
File "pyarrow/array.pxi", line 256, in pyarrow.lib.array
File "pyarrow/array.pxi", line 118, in pyarrow.lib._handle_arrow_array_protocol
File "/prime-environments/.venv/lib/python3.12/site-packages/datasets/arrow_writer.py", line 244, in __arrow_array__
out = pa.array(cast_to_python_objects(data, only_1d_for_numpy=True))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "pyarrow/array.pxi", line 375, in pyarrow.lib.array
File "pyarrow/array.pxi", line 46, in pyarrow.lib._sequence_to_array
File "pyarrow/error.pxi", line 155, in pyarrow.lib.pyarrow_internal_check_status
File "pyarrow/error.pxi", line 92, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: cannot mix struct and non-struct, non-null values