ForwardedEvents.evtx - Evtxecmd.exe processing errors

[BeagleDave]

You have done the community a huge service... This is a great utility.

I have, however, found what may be an interesting edge case. In rolling out a Windows Event Collection/Forwarding (WEC/WEF) infrastructure, I attempted to use your utility to dump the contents of an exemplar forwarded events log. Logging was set to archive and roll the ForwardedEvents log. The file size was manipulated so that I could produce a reasonably sized archive file and eliminate the possibility of me corrupting the event log file. The attached file was created and rolled by the system as part of normal log processing.
I run the following:
PS C:\bin\EvtxExplorer> ./evtxecmd.exe -f e:\workspace\Archive-ForwardedEvents-test.evtx --csv e:\workspace --debug
Version is: EvtxECmd version 0.5.2.0
I am getting the following error:
Record error at offset 0x1200, record #: 127638931 error: Specified argument was out of the range of valid values.
Parameter name: Value Type NullType is not handled! Handle it!
Record error at offset 0x2CE0, record #: 127638932 error: Index was out of range. Must be non-negative and less than the size of the collection.

I have attached the file in question.
Archive-ForwardedEvents-test.zip

Thanks!

Dave Crawford
D.S. Crawford
Information Security Office
California State University, Sacramento
6000 J Street, Sacramento CA 95819
Phone: (916) 278-1998
david.crawford@csus.edu

[EricZimmerman]

so is this just one log that fails, or all forwarded events fail?

[BeagleDave]

Eric Thanks for getting back! The file that I provided is an exemplar... The evtxecmd.exe utility is failing on _all_ the ForwardedEvents.evtx logs on my WEC server... I can send other example ForwardedEvents.evtx logs that have been rolled, if you would like. Dave C.

…

---------- Original Message ---------- From: Eric <notifications@github.com> Date: November 20, 2019 at 11:57 AM so is this just one log that fails, or all forwarded events fail? —You are receiving this because you authored the thread.Reply to this email directly, view it on GitHub, or unsubscribe.  

[EricZimmerman]

ok i see what is going on.

there are no templates defined for the log. ill add support for this situation asap. first time seeing it

[EricZimmerman]

i would love more sample files for my unit tests tho

[BeagleDave]

Eric:
The files are too big... Here's a link to a copy of a running log from today that I dumped on OneDrive:
https://mysacstate-my.sharepoint.com/:u:/g/personal/david_crawford_csus_edu/EZGlRmi_WmNKpOWlQ-2T00QBvvIEnWlWV8oD2cpysXP0Ew?e=ZbuvYa
The link will expire at the end of the month.
Dave C.

[BeagleDave]

I'm rolling the logs at 4 Gb... This one was half full when I grabbed it. Let me know if you would like any more exemplar files.
Thanks!
Dave C.

[EricZimmerman]

downloading

[robertstrom]

Hello,

I believe that I am having the same issue. I have Forwarded event logs from a lab environment. The EVTX file is about 2GB. I am also seeing the error on every event in the ForwardeEvents.evtx file.

Do you need additional sample logs?

FYI - just downloaded what I believe is the most current version , 0.6.0.0, dated 2/6/2020 and I am still seeing this error.

Thanks for all the amazing tools!

Robert

[CluelessAtCoding]

Hi Eric,

Firstly, thanks for making your tools available.

Secondly, I have just encountered this issue when trying to process a 10GB ForwardedEvents.evtx file from a WEC Server.

Thinking it was down to the file size I managed to create a smaller evtx file (2MB), using wevtutil, from the 10GB file which contained just the event ID I was initially after; but that has the same issue.

Debug Output:

EvtxECmd version 0.6.5.0

Author: Eric Zimmerman (saericzimmerman@gmail.com)
https://github.com/EricZimmerman/evtx

Command line: -f c:\path_to_evtx\test2.evtx --csv E:\WorkingFiles\SOURCE_SERVER_FWD_ONLY --csvf SOURCE_SERVER_FWD_1101_Only.csv --debug

Warning: Administrator privileges not found!

CSV output will be saved to 'E:\WorkingFiles\SOURCE_SERVER_FWD_1101_Only.csv'

Loading maps from 'C:\Utils\EricZimmerman\EvtxECmd\Maps'
'adPWDManager_adPWDManager_110.map' is valid. Adding to maps...
'Application_Application-Error_1000.map' is valid. Adding to maps...
'Application_Application-Hang_1002.map' is valid. Adding to maps...

...

'Windows-PowerShell_PowerShell_400.map' is valid. Adding to maps...
'Windows-PowerShell_PowerShell_403.map' is valid. Adding to maps...
'Windows-PowerShell_PowerShell_600.map' is valid. Adding to maps...
Maps loaded: 366

Processing 'c:\path_to_evtx\test2.evtx'...
Chunk count: 23, Iterating records...
Processing chunk at offset 0x1000. Events found so far: 0
Record error at offset 0x1200, record #: 1 error: Specified argument was out of the range of valid values.
Parameter name: Value Type NullType is not handled! Handle it!
Record error at offset 0x1B10, record #: 2 error: Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: startIndex

...

Record error at offset 0x11200, record #: 38 error: Specified argument was out of the range of valid values.
Parameter name: Value Type NullType is not handled! Handle it!
Record error at offset 0x11AF0, record #: 39 error: Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: startIndex

...

Parameter name: startIndex
Record error at offset 0x16A4F0, record #: 851 error: Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: startIndex
Record error at offset 0x16AB78, record #: 852 error: Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: startIndex
Record error at offset 0x16B200, record #: 853 error: Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: startIndex
Processing chunk at offset 0x171000. Events found so far: 0
Processing chunk at offset 0x181000. Events found so far: 0
Processing chunk at offset 0x191000. Events found so far: 0
Processing chunk at offset 0x1A1000. Events found so far: 0
Processing chunk at offset 0x1B1000. Events found so far: 0
Processing chunk at offset 0x1C1000. Events found so far: 0
Processing chunk at offset 0x1D1000. Events found so far: 0
Processing chunk at offset 0x1E1000. Events found so far: 0
Processing chunk at offset 0x1F1000. Events found so far: 0
Processing chunk at offset 0x201000. Events found so far: 0

Event log details
Flags: None
Chunk count: 23
Stored/Calculated CRC: 80C40644/80C40644
Earliest timestamp:
Latest timestamp:
Total event log records found: 0

Records included: 0 Errors: 853 Events dropped: 0

Errors
Record #1: Error: Specified argument was out of the range of valid values.
Parameter name: Value Type NullType is not handled! Handle it!
Record #2: Error: Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: startIndex
Record #3: Error: Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: startIndex

...

Record #850: Error: Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: startIndex
Record #851: Error: Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: startIndex
Record #852: Error: Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: startIndex
Record #853: Error: Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: startIndex

Processed 1 file in 1.2681 seconds

Files with errors
'c:\path_to_evtx\test2.evtx' error count: 853

Other than the actual evtx file (which I dont think I can give you due to the data it contains) , is there anything else I can provide to help you resolve this issue ?

Thanks again

Paul

[EricZimmerman]

I've seen this before with forwarded events. I'd need some sample logs so I can debug it tho

[CluelessAtCoding]

I've seen this before with forwarded events. I'd need some sample logs so I can debug it tho

Hi Eric, I have emailed you a sample. I hope it helps.

[EricZimmerman]

Ok great. I'll take a look asap

[BeagleDave]

Eric
We have revived our WEC/WEF infrastructure, and I noticed that EvtxECmd still has problems processing the forwarded events log.
I can provide exemplar of one of the archived event logs. Please note that this is a multi-volume 7-Zip file... Please delete the .txt that was added to enable me to upload this file type. Archive-ForwardedEvents-2024-03-19-05-57-01-714_2.7z.002.txt
Archive-ForwardedEvents-2024-03-19-05-57-01-714_2.7z.003.txt
Archive-ForwardedEvents-2024-03-19-05-57-01-714_2.7z.001.txt

Here are the initial error messages. Please note that nothing is being written to the output file.

PS C:\bin\EvtxECmd> .\EvtxECmd.exe -f "d:\test\Archive-ForwardedEvents-2024-03-19-18-36-47-754.evtx" --csv "c:\test\wec_wef" --csvf wec_wef.csv
EvtxECmd version 1.5.0.0

Author: Eric Zimmerman (saericzimmerman@gmail.com)
https://github.com/EricZimmerman/evtx

Command line: -f d:\test\Archive-ForwardedEvents-2024-03-19-18-36-47-754.evtx --csv c:\test\wec_wef --csvf wec_wef.csv

CSV output will be saved to c:\test\wec_wef\wec_wef.csv

Maps loaded: 423

Processing d:\test\Archive-ForwardedEvents-2024-03-19-18-36-47-754.evtx...
Chunk count: 31,130, Iterating records...
Record error at offset 0x1200, record #: 18292489156 error: Specified argument was out of the range of valid values.
Parameter name: Value Type NullType is not handled! Handle it!
System.ArgumentOutOfRangeException: Specified argument was out of the range of valid values.
Parameter name: Value Type NullType is not handled! Handle it!
at evtx.Tags.Value..ctor(Int64 recordPosition, BinaryReader dataStream, ChunkInfo chunk)
at evtx.Tags.TagBuilder.BuildTag(Int64 recordPosition, BinaryReader dataStream, ChunkInfo chunk)
at evtx.Tags.OpenStartElementTag..ctor(Int64 recordPosition, BinaryReader dataStream, ChunkInfo chunk, Boolean hasAttribute)
at evtx.Tags.TagBuilder.BuildTag(Int64 recordPosition, BinaryReader dataStream, ChunkInfo chunk)
at evtx.EventRecord..ctor(BinaryReader recordData, Int32 recordPosition, ChunkInfo chunk)
at evtx.ChunkInfo..ctor(Byte[] chunkBytes, Int64 absoluteOffset, Int32 chunkNumber)

Really appreciate your wizardry in providing this tool to the community!

Dave Crawford