-
Notifications
You must be signed in to change notification settings - Fork 18
Description
Using TileDB-VCF on Windows 11:
tiledb 2.20.1 conda-forge
tiledbvcf-py 0.29.0 tiledb
- when executing: ds.ingest_samples(sample_uris=["s3://tiledb-inc-demo-data/examples/notebooks/vcfs/1kg-dragen/HG00096_chr21.gvcf.gz"], threads=1) - see 'THREADS=1 - log' below, all fine
- when executing: ds.ingest_samples(sample_uris=["s3://tiledb-inc-demo-data/examples/notebooks/vcfs/1kg-dragen/HG00096_chr21.gvcf.gz"]) - see 'THREADS>1 - log' below, CreateFile exception - file in use
Might have been resolved in version 0.36.0 but I see Windows is still on 0.29.0.
Any clues? Thanks!
THREADS=1 - log:
[2024-12-16 11:14:52.702] [tiledb-vcf] [Process: 326516] [Thread: 320468] [debug] Lookup member name='data' in group uri='data/vcf/tiledb'
[2024-12-16 11:14:52.703] [tiledb-vcf] [Process: 326516] [Thread: 320468] [debug] Found group member name='data' uri='file:///C:/Play/data/vcf/tiledb/data'
[2024-12-16 11:14:52.703] [tiledb-vcf] [Process: 326516] [Thread: 320468] [debug] Open 'data' using group uri 'file:///C:/Play/data/vcf/tiledb/data'
[2024-12-16 11:14:52.704] [tiledb-vcf] [Process: 326516] [Thread: 320468] [debug] Lookup member name='vcf_headers' in group uri='data/vcf/tiledb'
[2024-12-16 11:14:52.705] [tiledb-vcf] [Process: 326516] [Thread: 320468] [debug] Found group member name='vcf_headers' uri='file:///C:/Play/data/vcf/tiledb/metadata/vcf_headers'
[2024-12-16 11:14:52.705] [tiledb-vcf] [Process: 326516] [Thread: 320468] [debug] Open 'vcf_headers' using group uri 'file:///C:/Play/data/vcf/tiledb/metadata/vcf_headers'
[2024-12-16 11:14:52.707] [tiledb-vcf] [Process: 326516] [Thread: 320468] [info] Memory budget: total=48876 MiB = tiledb=4096 MiB + input=10 MiB + output=22385 MiB + stats=22385 MiB
[2024-12-16 11:14:52.707] [tiledb-vcf] [Process: 326516] [Thread: 320468] [info] Input buffers = 1 threads * 10 samples * 1 MiB
[2024-12-16 11:14:52.707] [tiledb-vcf] [Process: 326516] [Thread: 320468] [info] Output buffers = 1 threads * 22385 MiB (flush = 16788 MiB)
[2024-12-16 11:14:52.709] [tiledb-vcf] [Process: 326516] [Thread: 320468] [debug] Lookup member name='data' in group uri='data/vcf/tiledb'
[2024-12-16 11:14:52.710] [tiledb-vcf] [Process: 326516] [Thread: 320468] [debug] Found group member name='data' uri='file:///C:/Play/data/vcf/tiledb/data'
[2024-12-16 11:14:52.710] [tiledb-vcf] [Process: 326516] [Thread: 320468] [debug] Open 'data' using group uri 'file:///C:/Play/data/vcf/tiledb/data'
[2024-12-16 11:14:52.711] [tiledb-vcf] [Process: 326516] [Thread: 320468] [debug] Lookup member name='vcf_headers' in group uri='data/vcf/tiledb'
[2024-12-16 11:14:52.712] [tiledb-vcf] [Process: 326516] [Thread: 320468] [debug] Found group member name='vcf_headers' uri='file:///C:/Play/data/vcf/tiledb/metadata/vcf_headers'
[2024-12-16 11:14:52.712] [tiledb-vcf] [Process: 326516] [Thread: 320468] [debug] Open 'vcf_headers' using group uri 'file:///C:/Play/data/vcf/tiledb/metadata/vcf_headers'
[2024-12-16 11:14:52.716] [tiledb-vcf] [Process: 326516] [Thread: 320468] [debug] AlleleCount: Open array 'file:///C:/Play/data/vcf/tiledb/allele_count'
[2024-12-16 11:14:52.717] [tiledb-vcf] [Process: 326516] [Thread: 320468] [debug] VariantStats: Open array 'file:///C:/Play/data/vcf/tiledb/variant_stats'
[2024-12-16 11:14:54.025] [tiledb-vcf] [Process: 326516] [Thread: 320468] [debug] Initialization completed in 1.318 seconds.
[2024-12-16 11:14:54.566] [tiledb-vcf] [Process: 326516] [Thread: 320468] [debug] Reading VCF header vfs://s3://tiledb-inc-demo-data/examples/notebooks/vcfs/1kg-dragen/HG00096_chr21.gvcf.gz
[2024-12-16 11:14:55.100] [tiledb-vcf] [Process: 326516] [Thread: 320468] [debug] Reading VCF header vfs://s3://tiledb-inc-demo-data/examples/notebooks/vcfs/1kg-dragen/HG00096_chr21.gvcf.gz
[2024-12-16 11:14:55.546] [tiledb-vcf] [Process: 326516] [Thread: 320468] [debug] Reading VCF header vfs://s3://tiledb-inc-demo-data/examples/notebooks/vcfs/1kg-dragen/HG00096_chr21.gvcf.gz
[2024-12-16 11:14:56.006] [tiledb-vcf] [Process: 326516] [Thread: 320468] [debug] Output buffer records = 34381824
[2024-12-16 11:14:56.006] [tiledb-vcf] [Process: 326516] [Thread: 320468] [debug] Contig chr21: task size = 46.709.984 tasks = 1
[2024-12-16 11:14:56.027] [tiledb-vcf] [Process: 326516] [Thread: 320468] [debug] Current contig = Next contig =
[2024-12-16 11:14:56.027] [tiledb-vcf] [Process: 326516] [Thread: 308532] [debug] WriteWorker4: parse chr21:0-46709983
[2024-12-16 11:14:56.507] [tiledb-vcf] [Process: 326516] [Thread: 308532] [debug] AlleleCount: new contig = chr21
[2024-12-16 11:14:56.508] [tiledb-vcf] [Process: 326516] [Thread: 308532] [debug] VariantStats: new contig = chr21
[2024-12-16 11:15:30.625] [tiledb-vcf] [Process: 326516] [Thread: 320468] [debug] Recorded 2324491 cells from chr21:5030082-46699955 (task 1 / 1)
[2024-12-16 11:15:30.626] [tiledb-vcf] [Process: 326516] [Thread: 320468] [debug] AlleleCount: flushing 80602 records from chr21:5030087-46699312
[2024-12-16 11:15:30.664] [tiledb-vcf] [Process: 326516] [Thread: 320468] [debug] VariantStats: flushing 2363718 records from chr21:5030082-46699946
[2024-12-16 11:15:31.350] [tiledb-vcf] [Process: 326516] [Thread: 320468] [debug] Worker 0: Drained 372 anchors.
[2024-12-16 11:15:31.351] [tiledb-vcf] [Process: 326516] [Thread: 320468] [info] Ingestion rate = 65805.307 records/sec (VmRSS = NA)
[2024-12-16 11:15:31.351] [tiledb-vcf] [Process: 326516] [Thread: 320468] [debug] Finalizing last contig batch of [chr21, chr21]
[2024-12-16 11:15:31.351] [tiledb-vcf] [Process: 326516] [Thread: 320468] [debug] AlleleCount: Finalize query with 80602 records
[2024-12-16 11:15:31.389] [tiledb-vcf] [Process: 326516] [Thread: 320468] [debug] AlleleCount: fragment_num = 1 uri = file:///C:/Play/data/vcf/tiledb/allele_count/__fragments/__1734344130626_1734344130626_04458c2356dd461cb1b4323eb00b4f3d_21 samples = HG00096
[2024-12-16 11:15:31.390] [tiledb-vcf] [Process: 326516] [Thread: 320468] [debug] VariantStats: Finalize query with 2363718 records
[2024-12-16 11:15:31.426] [tiledb-vcf] [Process: 326516] [Thread: 320468] [debug] VariantStats: fragment_num = 1 uri = file:///C:/Play/data/vcf/tiledb/variant_stats/__fragments/__1734344130667_1734344130667_79989b5ec1db47568dddda8d999b03e9_21 samples = HG00096
[2024-12-16 11:15:31.427] [tiledb-vcf] [Process: 326516] [Thread: 320468] [debug] Write 372 anchor records.
[2024-12-16 11:15:31.516] [tiledb-vcf] [Process: 326516] [Thread: 320468] [debug] Making sure all finalize tasks completed...
[2024-12-16 11:15:31.596] [tiledb-vcf] [Process: 326516] [Thread: 320468] [debug] All finalize tasks successfully completed. Waited for 0.0807968 sec.
[2024-12-16 11:15:31.597] [tiledb-vcf] [Process: 326516] [Thread: 320468] [debug] AlleleCount: Close array
[2024-12-16 11:15:31.600] [tiledb-vcf] [Process: 326516] [Thread: 320468] [debug] VariantStats: Close array
[2024-12-16 11:15:31.603] [tiledb-vcf] [Process: 326516] [Thread: 320468] [info] Done. Ingested 2.324.491 records (+ 372 anchors) from 1 samples in 38.897 seconds = 59760.7 records/sec
[2024-12-16 11:15:31.604] [tiledb-vcf] [Process: 326516] [Thread: 320468] [info] QACheck: [PASS] Total records ingested (2324491) == total records in VCF files (2324491)
THREADS>1 - log:
[2024-12-16 11:16:45.659] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Lookup member name='data' in group uri='data/vcf/tiledb'
[2024-12-16 11:16:45.660] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Found group member name='data' uri='file:///C:/Play/data/vcf/tiledb/data'
[2024-12-16 11:16:45.660] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Open 'data' using group uri 'file:///C:/Play/data/vcf/tiledb/data'
[2024-12-16 11:16:45.667] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Lookup member name='vcf_headers' in group uri='data/vcf/tiledb'
[2024-12-16 11:16:45.668] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Found group member name='vcf_headers' uri='file:///C:/Play/data/vcf/tiledb/metadata/vcf_headers'
[2024-12-16 11:16:45.668] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Open 'vcf_headers' using group uri 'file:///C:/Play/data/vcf/tiledb/metadata/vcf_headers'
[2024-12-16 11:16:45.669] [tiledb-vcf] [Process: 325032] [Thread: 321184] [info] Memory budget: total=48876 MiB = tiledb=4096 MiB + input=320 MiB + output=22230 MiB + stats=22230 MiB
[2024-12-16 11:16:45.670] [tiledb-vcf] [Process: 325032] [Thread: 321184] [info] Input buffers = 32 threads * 10 samples * 1 MiB
[2024-12-16 11:16:45.670] [tiledb-vcf] [Process: 325032] [Thread: 321184] [info] Output buffers = 32 threads * 694 MiB (flush = 521 MiB)
[2024-12-16 11:16:45.673] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Lookup member name='data' in group uri='data/vcf/tiledb'
[2024-12-16 11:16:45.674] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Found group member name='data' uri='file:///C:/Play/data/vcf/tiledb/data'
[2024-12-16 11:16:45.674] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Open 'data' using group uri 'file:///C:/Play/data/vcf/tiledb/data'
[2024-12-16 11:16:45.675] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Lookup member name='vcf_headers' in group uri='data/vcf/tiledb'
[2024-12-16 11:16:45.676] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Found group member name='vcf_headers' uri='file:///C:/Play/data/vcf/tiledb/metadata/vcf_headers'
[2024-12-16 11:16:45.676] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Open 'vcf_headers' using group uri 'file:///C:/Play/data/vcf/tiledb/metadata/vcf_headers'
[2024-12-16 11:16:45.679] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] AlleleCount: Open array 'file:///C:/Play/data/vcf/tiledb/allele_count'
[2024-12-16 11:16:45.680] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] VariantStats: Open array 'file:///C:/Play/data/vcf/tiledb/variant_stats'
[2024-12-16 11:16:46.983] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Initialization completed in 1.314 seconds.
[2024-12-16 11:16:47.440] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Reading VCF header vfs://s3://tiledb-inc-demo-data/examples/notebooks/vcfs/1kg-dragen/HG00096_chr21.gvcf.gz
[2024-12-16 11:16:47.886] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Reading VCF header vfs://s3://tiledb-inc-demo-data/examples/notebooks/vcfs/1kg-dragen/HG00096_chr21.gvcf.gz
[2024-12-16 11:16:48.334] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Reading VCF header vfs://s3://tiledb-inc-demo-data/examples/notebooks/vcfs/1kg-dragen/HG00096_chr21.gvcf.gz
[2024-12-16 11:16:48.786] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Reading VCF header vfs://s3://tiledb-inc-demo-data/examples/notebooks/vcfs/1kg-dragen/HG00096_chr21.gvcf.gz
[2024-12-16 11:16:49.243] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Reading VCF header vfs://s3://tiledb-inc-demo-data/examples/notebooks/vcfs/1kg-dragen/HG00096_chr21.gvcf.gz
[2024-12-16 11:16:49.706] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Reading VCF header vfs://s3://tiledb-inc-demo-data/examples/notebooks/vcfs/1kg-dragen/HG00096_chr21.gvcf.gz
[2024-12-16 11:16:50.158] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Reading VCF header vfs://s3://tiledb-inc-demo-data/examples/notebooks/vcfs/1kg-dragen/HG00096_chr21.gvcf.gz
[2024-12-16 11:16:50.614] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Reading VCF header vfs://s3://tiledb-inc-demo-data/examples/notebooks/vcfs/1kg-dragen/HG00096_chr21.gvcf.gz
[2024-12-16 11:16:51.064] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Reading VCF header vfs://s3://tiledb-inc-demo-data/examples/notebooks/vcfs/1kg-dragen/HG00096_chr21.gvcf.gz
[2024-12-16 11:16:51.524] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Reading VCF header vfs://s3://tiledb-inc-demo-data/examples/notebooks/vcfs/1kg-dragen/HG00096_chr21.gvcf.gz
[2024-12-16 11:16:51.985] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Reading VCF header vfs://s3://tiledb-inc-demo-data/examples/notebooks/vcfs/1kg-dragen/HG00096_chr21.gvcf.gz
[2024-12-16 11:16:52.450] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Reading VCF header vfs://s3://tiledb-inc-demo-data/examples/notebooks/vcfs/1kg-dragen/HG00096_chr21.gvcf.gz
[2024-12-16 11:16:52.912] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Reading VCF header vfs://s3://tiledb-inc-demo-data/examples/notebooks/vcfs/1kg-dragen/HG00096_chr21.gvcf.gz
[2024-12-16 11:16:53.364] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Reading VCF header vfs://s3://tiledb-inc-demo-data/examples/notebooks/vcfs/1kg-dragen/HG00096_chr21.gvcf.gz
[2024-12-16 11:16:53.825] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Reading VCF header vfs://s3://tiledb-inc-demo-data/examples/notebooks/vcfs/1kg-dragen/HG00096_chr21.gvcf.gz
[2024-12-16 11:16:54.278] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Reading VCF header vfs://s3://tiledb-inc-demo-data/examples/notebooks/vcfs/1kg-dragen/HG00096_chr21.gvcf.gz
[2024-12-16 11:16:54.740] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Reading VCF header vfs://s3://tiledb-inc-demo-data/examples/notebooks/vcfs/1kg-dragen/HG00096_chr21.gvcf.gz
[2024-12-16 11:16:55.223] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Reading VCF header vfs://s3://tiledb-inc-demo-data/examples/notebooks/vcfs/1kg-dragen/HG00096_chr21.gvcf.gz
[2024-12-16 11:16:55.681] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Reading VCF header vfs://s3://tiledb-inc-demo-data/examples/notebooks/vcfs/1kg-dragen/HG00096_chr21.gvcf.gz
[2024-12-16 11:16:56.140] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Reading VCF header vfs://s3://tiledb-inc-demo-data/examples/notebooks/vcfs/1kg-dragen/HG00096_chr21.gvcf.gz
[2024-12-16 11:16:56.587] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Reading VCF header vfs://s3://tiledb-inc-demo-data/examples/notebooks/vcfs/1kg-dragen/HG00096_chr21.gvcf.gz
[2024-12-16 11:16:57.041] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Reading VCF header vfs://s3://tiledb-inc-demo-data/examples/notebooks/vcfs/1kg-dragen/HG00096_chr21.gvcf.gz
[2024-12-16 11:16:57.493] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Reading VCF header vfs://s3://tiledb-inc-demo-data/examples/notebooks/vcfs/1kg-dragen/HG00096_chr21.gvcf.gz
[2024-12-16 11:16:58.509] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Reading VCF header vfs://s3://tiledb-inc-demo-data/examples/notebooks/vcfs/1kg-dragen/HG00096_chr21.gvcf.gz
[2024-12-16 11:16:59.059] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Reading VCF header vfs://s3://tiledb-inc-demo-data/examples/notebooks/vcfs/1kg-dragen/HG00096_chr21.gvcf.gz
[2024-12-16 11:16:59.529] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Reading VCF header vfs://s3://tiledb-inc-demo-data/examples/notebooks/vcfs/1kg-dragen/HG00096_chr21.gvcf.gz
[2024-12-16 11:17:00.000] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Reading VCF header vfs://s3://tiledb-inc-demo-data/examples/notebooks/vcfs/1kg-dragen/HG00096_chr21.gvcf.gz
[2024-12-16 11:17:00.464] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Reading VCF header vfs://s3://tiledb-inc-demo-data/examples/notebooks/vcfs/1kg-dragen/HG00096_chr21.gvcf.gz
[2024-12-16 11:17:00.938] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Reading VCF header vfs://s3://tiledb-inc-demo-data/examples/notebooks/vcfs/1kg-dragen/HG00096_chr21.gvcf.gz
[2024-12-16 11:17:01.398] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Reading VCF header vfs://s3://tiledb-inc-demo-data/examples/notebooks/vcfs/1kg-dragen/HG00096_chr21.gvcf.gz
[2024-12-16 11:17:01.888] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Reading VCF header vfs://s3://tiledb-inc-demo-data/examples/notebooks/vcfs/1kg-dragen/HG00096_chr21.gvcf.gz
[2024-12-16 11:17:02.343] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Reading VCF header vfs://s3://tiledb-inc-demo-data/examples/notebooks/vcfs/1kg-dragen/HG00096_chr21.gvcf.gz
[2024-12-16 11:17:02.805] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Reading VCF header vfs://s3://tiledb-inc-demo-data/examples/notebooks/vcfs/1kg-dragen/HG00096_chr21.gvcf.gz
[2024-12-16 11:17:03.294] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Reading VCF header vfs://s3://tiledb-inc-demo-data/examples/notebooks/vcfs/1kg-dragen/HG00096_chr21.gvcf.gz
[2024-12-16 11:17:03.763] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Output buffer records = 1067008
[2024-12-16 11:17:03.763] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Contig chr21: task size = 16.080.916 tasks = 3
[2024-12-16 11:17:03.783] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Current contig = Next contig = chr21
[2024-12-16 11:17:03.783] [tiledb-vcf] [Process: 325032] [Thread: 325108] [debug] WriteWorker4: parse chr21:0-16080915
[2024-12-16 11:17:03.783] [tiledb-vcf] [Process: 325032] [Thread: 321988] [debug] WriteWorker4: parse chr21:16080916-32161831
[2024-12-16 11:17:03.783] [tiledb-vcf] [Process: 325032] [Thread: 310844] [debug] WriteWorker4: parse chr21:32161832-46709983
[2024-12-16 11:17:04.263] [tiledb-vcf] [Process: 325032] [Thread: 325108] [debug] AlleleCount: new contig = chr21
[2024-12-16 11:17:04.263] [tiledb-vcf] [Process: 325032] [Thread: 325108] [debug] VariantStats: new contig = chr21
[2024-12-16 11:17:04.754] [tiledb-vcf] [Process: 325032] [Thread: 321988] [debug] AlleleCount: new contig = chr21
[2024-12-16 11:17:04.754] [tiledb-vcf] [Process: 325032] [Thread: 321988] [debug] VariantStats: new contig = chr21
[2024-12-16 11:17:05.099] [tiledb-vcf] [Process: 325032] [Thread: 310844] [debug] AlleleCount: new contig = chr21
[2024-12-16 11:17:05.099] [tiledb-vcf] [Process: 325032] [Thread: 310844] [debug] VariantStats: new contig = chr21
[2024-12-16 11:17:10.378] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Recorded 386659 cells from chr21:5030082-16080909 (task 1 / 3)
[2024-12-16 11:17:10.379] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] AlleleCount: flushing 20775 records from chr21:5030087-16080745
[2024-12-16 11:17:10.387] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] VariantStats: flushing 397915 records from chr21:5030082-16080909
[2024-12-16 11:17:10.440] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Worker 0: Drained 26 anchors.
[2024-12-16 11:17:10.440] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] AlleleCount: Finalize query with 20775 records
[2024-12-16 11:17:10.466] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] AlleleCount: fragment_num = 1 uri = file:///C:/Play/data/vcf/tiledb/allele_count/__fragments/__1734344230379_1734344230379_29df30bd8f634524a08aa5e9fb2e7eea_21 samples = HG00096
[2024-12-16 11:17:10.466] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] VariantStats: Finalize query with 397915 records
[2024-12-16 11:17:10.494] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] VariantStats: fragment_num = 1 uri = file:///C:/Play/data/vcf/tiledb/variant_stats/__fragments/__1734344230388_1734344230388_f8e13ba5ecf14b5ea05a24153431405a_21 samples = HG00096
[2024-12-16 11:17:10.494] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] Current contig = chr21 Next contig = chr21
[2024-12-16 11:17:18.917] [tiledb-vcf] [Process: 325032] [Thread: 321184] [error] TileDB-VCF exception: [TileDB::IO] Error: Cannot write to file 'C:\Play\data\vcf\tiledb\data__fragments__1734344230208_1734344230208_8141b7595cee4f91bea4e46761e1c315_21\a1.tdb'; File opening error CreateFile GetLastError 32 (0x00000020): The process cannot access the file because it is being used by another process.
Traceback (most recent call last):
File "c:\Play\app\vcf\main.py", line 44, in
ds.ingest_samples(sample_uris=["s3://tiledb-inc-demo-data/examples/notebooks/vcfs/1kg-dragen/HG00096_chr21.gvcf.gz"])
File "c:\Users\Peter Bossier\miniconda3\Lib\site-packages\tiledbvcf\dataset.py", line 858, in ingest_samples
self.writer.ingest_samples()
RuntimeError: TileDB-VCF exception: [TileDB::IO] Error: Cannot write to file 'C:\Play\data\vcf\tiledb\data__fragments__1734344230208_1734344230208_8141b7595cee4f91bea4e46761e1c315_21\a1.tdb'; File opening error CreateFile GetLastError 32 (0x00000020): The process cannot access the file because it is being used by another process.
[2024-12-16 11:17:18.929] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] AlleleCount: Close array
[2024-12-16 11:17:18.933] [tiledb-vcf] [Process: 325032] [Thread: 321184] [debug] VariantStats: Close array