new eval schema, contributor guide, schema fix testing · evaleval/every_eval_ever@2891f09

Triggered via pull request November 26, 2025 16:58

akornilo

opened #10

various-follow-ups-ak

Status Failure

Total duration 13s

Artifacts –

validate-data.yml

on: pull_request

10 errors

ValidationError: data/hfopenllm_v2/mistral/yam-peleg/Hebrew-Mistral-7B-200K/4d45347d-4491-4d7b-9abe-02c42974f520.json#L0

ValidationError: {'score': 0.17698041197356346} is not of type 'string'

ValidationError: data/hfopenllm_v2/arcee-ai/Arcee-Blitz/01e8e033-1aa9-42e2-85d8-b7974d0c9e23.json#L0

ValidationError: {'score': 0.5543435861292482} is not of type 'string'

ValidationError: data/hfopenllm_v2/gmonsoon/SahabatAI-MediChatIndo-8B-v1/61543864-320f-41ef-889d-7c0e95a229bd.json#L0

ValidationError: {'score': 0.41628323958208663} is not of type 'string'

ValidationError: data/hfopenllm_v2/Sakalti/SakalFusion-7B-Alpha/2329f6f2-228a-400b-9b2d-4ad6dd278b79.json#L0

ValidationError: {'score': 0.5289653674472622} is not of type 'string'

ValidationError: data/hfopenllm_v2/meta/dfurman/Llama-3-8B-Orpo-v0.1/0a6a3c2b-c0f5-44c7-9ac2-e278a303197e.json#L0

ValidationError: {'score': 0.3000039894147528} is not of type 'string'

ValidationError: data/hfopenllm_v2/marcuscedricridia/pre-cursa-o1-v1.6/50627b31-a8d4-401a-8449-5f33cfb17893.json#L0

ValidationError: {'score': 0.7527549125209998} is not of type 'string'

ValidationError: data/hfopenllm_v2/jaspionjader/f-6-8b/2a71c7d7-8ae6-45e7-ab7f-54f7d31dd131.json#L0

ValidationError: {'score': 0.48460196722474147} is not of type 'string'

ValidationError: data/hfopenllm_v2/allknowingroger/MixTAO-19B-pass/275fb96e-4779-479b-937b-f5db6aa530ea.json#L0

ValidationError: {'score': 0.3814368098866563} is not of type 'string'

ValidationError: data/hfopenllm_v2/sthenno-com/miscii-14b-0218/f73b09b4-020d-49fd-8ede-6a690088be94.json#L0

ValidationError: {'score': 0.7655941790006073} is not of type 'string'

ValidationError: data/hfopenllm_v2/godlikehhd/alpaca_data_ifd_min_2600/5561b7bd-bd90-445c-b969-8d400e99e629.json#L0

ValidationError: {'score': 0.3749673089624419} is not of type 'string'