[Bug]: An assertion occurred: ASSERT [DxeCore] edk2/MdeModulePkg/Core/Dxe/Mem/Pool.c(730): Tail->Signature == ((('p') | ('t' << 8)) | ((('a') | ('l' << 8)) << 16))

[HayatoTasaki]

Is there an existing issue for this?

• I have searched existing issues

Bug Type

• Firmware
• Tool
• Unit Test

What packages are impacted?

MdeModulePkg

Which targets are impacted by this bug?

RELEASE

Current Behavior

An assertion occurred:
ASSERT [DxeCore] edk2/MdeModulePkg/Core/Dxe/Mem/Pool.c(730): Tail->Signature == ((('p') | ('t' << 8)) | ((('a') | ('l' << 8)) << 16))

The frequency of occurrence is approximately once every 1 in 5000 boots.
The hardware is a custom ARM-based board.
Please advise on how to fix this issue.

Expected Behavior

The ASSERT [DxeCore] no longer occurs.

Steps To Reproduce

Reboot loop triggered by sudo reboot command.

Build Environment

- OS(s): Ubuntu 22.04
- Tool Chain(s): aarch64--glibc--stable-2022.08-1

Version Information

EDK2 version:
PACKAGE_NAME = MdeModulePkg
PACKAGE_VERSION = 0.98

Urgency

High

Are you going to fix this?

Someone else needs to fix it

Do you need maintainer feedback?

Maintainer feedback requested

Anything else?

No response

[HayatoTasaki]

We’d appreciate a response at your earliest convenience.

[lersek]

My opinion: FreePool detects a buffer overflow. The tail signature of an earlier pool allocation gets overwritten by the time it's released. This could originate anywhence. I suggest a debug patch for CoreFreePoolI: when Tail->Signature != POOL_TAIL_SIGNATURE, log a hexdump of Head->Size bytes starting from Head. The contents of the buffer may betray the origin of the data -- i.e., what component is responsible for filling (and overflowing) the buffer.

[HayatoTasaki]

Thank you for your response.

I will implement a process to add hexdump logging and run the tests again.

Before reporting this issue, I also applied a fix, and I’d like to share that as additional context.

I noticed that in the CoreConnectController() function in MdeModulePkg/Core/Dxe/Hand/DriverSupport.c, when ChildHandleCount is 0, executing CoreFreePool(ChildHandleBuffer); can sometimes trigger an ASSERT [DxeCore].

To address this, I modified the logic so that when ChildHandleCount is 0, the connection of the controller handle's children is skipped entirely (i.e., no memory allocation or deallocation is performed). After applying this change, the ASSERT no longer occurs at this location.

Q1. Is this an appropriate fix?

However, I encountered another ASSERT [DxeCore] in a different location.

In the CoreConnectSingleController() function in the same file, CoreFreePool(SortedDriverBindingProtocols); occasionally triggers an ASSERT.

Before the call to CoreFreePool(SortedDriverBindingProtocols);, I logged the address of SortedDriverBindingProtocols, and also logged the buffer address received by CoreFreePoolI(). When the ASSERT occurs, the addresses differ.

Log when ASSERT occurs:

DriverBindingHandleCount = 43 : New = 43 : NumberOfSortedDriverBindingProtocols = 43
SortedDriverBindingProtocols = 6080847128
HeadSize = 2896 : TailSize = 2896 : TailSignature = 0 0 0 4294967296
Buffer = 6080847896

Log when normal:

DriverBindingHandleCount = 1 : New = 1 : NumberOfSortedDriverBindingProtocols = 1
SortedDriverBindingProtocols = 6089299224
HeadSize = 48 : TailSize = 48 : TailSignature = 1818326128
Buffer = 6089299224
When the ASSERT occurs, it seems that the counter used during allocation and the Head->Size calculation do not match.

Q2. Based on this information, do you have any ideas that could help resolve the issue?

[lersek]

I noticed that in the CoreConnectController() function in MdeModulePkg/Core/Dxe/Hand/DriverSupport.c, when ChildHandleCount is 0, executing CoreFreePool(ChildHandleBuffer); can sometimes trigger an ASSERT [DxeCore].

To address this, I modified the logic so that when ChildHandleCount is 0, the connection of the controller handle's children is skipped entirely (i.e., no memory allocation or deallocation is performed). After applying this change, the ASSERT no longer occurs at this location.

Q1. Is this an appropriate fix?

Distinguishing ChildHandleCount==0 should not be necessary. MemoryAllocationLib.h states about AllocatePool: If AllocationSize is 0, then a valid buffer of 0 size is returned. Consequently, releasing ChildHandleBuffer with CoreFreePool should be valid, even if the allocation size was 0 bytes. (AllocatePool in the DxeCoreMemoryAllocationLib instance simply wraps CoreAllocatePool, so it should not be a problem that AllocatePool allocates but the less abstract CoreFreePool releases.)

In the CoreConnectSingleController() function in the same file, CoreFreePool(SortedDriverBindingProtocols); occasionally triggers an ASSERT.

Before the call to CoreFreePool(SortedDriverBindingProtocols);, I logged the address of SortedDriverBindingProtocols, and also logged the buffer address received by CoreFreePoolI(). When the ASSERT occurs, the addresses differ.

Log when ASSERT occurs:

DriverBindingHandleCount = 43 : New = 43 : NumberOfSortedDriverBindingProtocols = 43
SortedDriverBindingProtocols = 6080847128
HeadSize = 2896 : TailSize = 2896 : TailSignature = 0 0 0 4294967296
Buffer = 6080847896

Log when normal:

DriverBindingHandleCount = 1 : New = 1 : NumberOfSortedDriverBindingProtocols = 1
SortedDriverBindingProtocols = 6089299224
HeadSize = 48 : TailSize = 48 : TailSignature = 1818326128
Buffer = 6089299224

When the ASSERT occurs, it seems that the counter used during allocation and the Head->Size calculation do not match.

Q2. Based on this information, do you have any ideas that could help resolve the issue?

In the normal case, you have 1 element (one pointer) in SortedDriverBindingProtocols. The internal allocation size (48 bytes) seems consistent with that; SIZE_OF_POOL_HEAD is 20 bytes, sizeof (POOL_TAIL) is 16 bytes, 20+1*8+16=44, which, rounded up to a multiple of 8 bytes, yields 48. The tail signature matches too (1818326128 decimal is 0x6C617470, which is POOLPAGE_HEAD_SIGNATURE).

In the failing case, the 2896 bytes size looks weird. If you had indeed 43 pointers in SortedDriverBindingProtocols, I'd expect an internal allocation size of 20+43*8+16=380; 384 after aligning up. The tail signature looks corrupted too (although I don't understand why your log contains four values there, 0 0 0 4294967296).

The SortedDriverBindingProtocols pointer's value should never change between AllocatePool and CoreFreePool. In the failing case, it seems to shift by 0x300 bytes, which seems to "regular" for it to be random corruption. Still, I guess I'd sprinkle CoreConnectSingleController with a bunch of DEBUG messages, to narrow down where SortedDriverBindingProtocols changes.

[HayatoTasaki]

Distinguishing ChildHandleCount==0 should not be necessary. MemoryAllocationLib.h states about AllocatePool: If AllocationSize is 0, then a valid buffer of 0 size is returned. Consequently, releasing ChildHandleBuffer with CoreFreePool should be valid, even if the allocation size was 0 bytes. (AllocatePool in the DxeCoreMemoryAllocationLib instance simply wraps CoreAllocatePool, so it should not be a problem that AllocatePool allocates but the less abstract CoreFreePool releases.)

To prevent the ASSERT [DxeCore], is it problematic to add logic in MdeModulePkg/Core/Dxe/Hand/DriverSupport.c that skips memory allocation when ChildHandleCount is 0?
In CoreConnectSingleController(), there is also a return statement when DriverBindingHandleCount is 0, similar to the logic we added.
What is the difference between these two implementations?

[lersek]

Explicitly skipping memory allocation for zero bytes, plus the allocation-dependent logic, should not hurt, but it should not be necessary either. Even if the allocation + usage + release is there, the code should behave properly. Allocation of zero bytes is supposed to return a valid address, suitable for freeing. The logic in the middle should not dereference anything anyway (with the element count being zero). Therefore distinguishing the zero bytes allocation path could be considered a minor perf optimization (perhaps), at the cost of complicating the code. For making an actual symptom go away, the separate path can be considered a workaround at best. If the code fails when allocating zero bytes, then there's a problem underneath somewhere; possibly the MemoryAllocationLib instance at hand violating the contract for allocating zero bytes (I doubt that should be the problem though; it seems unlikely that the Dxe Core instance of this lib has such a corner case). I think it's more likely that something corrupts the memory structures.

[HayatoTasaki]

I’ve implemented hexdump logging and have been conducting tests, and so far no ASSERT has occurred.

To help guide further investigation, I’d like to ask:
Based on your experience or general knowledge, what kinds of factors could potentially lead to memory corruption in this kind of environment?

[lersek]

I wouldn't like to sound like I knew a better "catalog" of buffer overflows than anyone else. General C programming mistakes come to mind:

• string termination
• assuming a fixed size buffer when the data is in fact variable size
• integer overflow (multiplication result doesn't fit, or signed/unsigned confusion, or wrong assumption about magnitude of operands)
• missing input validation (network or user input)
• logic bugs in storage or network drivers

The fact that the symptom is hard to reproduce is discouraging (1 in 5000, in a reboot loop). It could be timing-related. In such cases we're tempted to capture verbose logs, and then work backwards in the logs, comparing the logs of successful vs. failed boots. Unfortunately, logging may introduce enough delay to mask the issue (if it is indeed timing-related).

We also have events and notification functions; it some driver doesn't manage the TPL properly (doesn't implement "locking" properly), some code could be re-entered that was never made reentrant. That would be an unsurprising source of memory corruption.

[HayatoTasaki]

We added log output for investigation purposes, but the ASSERT no longer occurred. Therefore, we modified the investigation to only output a memory dump when Tail->Signature != POOL_TAIL_SIGNATURE.
For reference, this condition is written in the file MdeModulePkg/Core/Dxe/Mem/Pool.c.

After approximately 100 reboots, an ASSERT in [DxeCore] occurred twice — once during the 79th reboot and again during the 88th reboot.

Is there anything you can identify from these dumps?
We are providing the memory dumps for both occurrences below:
The memory dump is performed starting from Head and spans Head->Size bytes

Memory dump from the 79th reboot (first ASSERT occurrence).txt

Memory dump from the 88th reboot (second ASSERT occurrence).txt

• Memory dump from the 79th reboot (first ASSERT occurrence)
  00000000: 70 68 64 30 13 10 2A 01 04 00 00 00 00 00 A0 F0
  00000010: 48 0B 00 00 00 00 00 00 98 FF 51 6C 01 00 00 00
  00000020: 18 EF 51 6C 01 00 00 00 98 EC 51 6C 01 00 00 00
  00000030: 18 1F 50 6C 01 00 00 00 18 1A 50 6C 01 00 00 00
  00000040: 18 9E 4F 6C 01 00 00 00 18 14 F5 6A 01 00 00 00
  00000050: 18 FE F3 6A 01 00 00 00 18 EF F3 6A 01 00 00 00
  00000060: 18 E9 F3 6A 01 00 00 00 98 BF F3 6A 01 00 00 00
  00000070: 98 B9 F3 6A 01 00 00 00 18 50 F3 6A 01 00 00 00
  00000080: 18 59 F3 6A 01 00 00 00 18 2E F3 6A 01 00 00 00
  00000090: 98 43 F3 6A 01 00 00 00 98 48 F3 6A 01 00 00 00
  000000A0: 98 3B F3 6A 01 00 00 00 18 23 F3 6A 01 00 00 00
  000000B0: 98 10 F3 6A 01 00 00 00 18 12 F3 6A 01 00 00 00
  000000C0: 98 2C F2 6A 01 00 00 00 18 26 F2 6A 01 00 00 00
  000000D0: 18 28 F2 6A 01 00 00 00 18 08 F3 6A 01 00 00 00
  000000E0: 98 EE F2 6A 01 00 00 00 98 E6 F2 6A 01 00 00 00
  000000F0: 18 E6 F2 6A 01 00 00 00 98 DE F2 6A 01 00 00 00
  00000100: 18 9F F2 6A 01 00 00 00 18 9C F2 6A 01 00 00 00
  00000110: 18 99 F2 6A 01 00 00 00 98 93 F2 6A 01 00 00 00
  00000120: 98 7D F2 6A 01 00 00 00 98 78 F2 6A 01 00 00 00
  00000130: 18 73 F2 6A 01 00 00 00 18 63 F2 6A 01 00 00 00
  00000140: 98 5B F2 6A 01 00 00 00 98 5C F2 6A 01 00 00 00
  00000150: 98 59 F2 6A 01 00 00 00 18 4C F2 6A 01 00 00 00
  00000160: 98 47 F2 6A 01 00 00 00 18 3E F2 6A 01 00 00 00
  00000170: 18 38 F2 6A 01 00 00 00 98 35 F2 6A 01 00 00 00
  00000180: 98 15 F2 6A 01 00 00 00 98 69 F1 6A 01 00 00 00
  00000190: 98 67 F1 6A 01 00 00 00 18 5A F1 6A 01 00 00 00
  000001A0: 98 54 F1 6A 01 00 00 00 18 0C F2 6A 01 00 00 00
  000001B0: 98 0E F2 6A 01 00 00 00 18 FC F1 6A 01 00 00 00
  000001C0: 98 F2 F1 6A 01 00 00 00 18 D8 F1 6A 01 00 00 00
  000001D0: 98 AE F1 6A 01 00 00 00 18 A7 F1 6A 01 00 00 00
  000001E0: 98 A1 F1 6A 01 00 00 00 18 98 F1 6A 01 00 00 00
  000001F0: 98 83 F1 6A 01 00 00 00 18 7E F1 6A 01 00 00 00
  00000200: 98 77 F1 6A 01 00 00 00 98 72 F1 6A 01 00 00 00
  00000210: 18 AA F0 6A 01 00 00 00 18 A4 F0 6A 01 00 00 00
  00000220: 98 A8 F0 6A 01 00 00 00 18 4B F1 6A 01 00 00 00
  00000230: 98 3E F1 6A 01 00 00 00 18 48 F1 6A 01 00 00 00
  00000240: 18 39 F1 6A 01 00 00 00 18 35 F1 6A 01 00 00 00
  00000250: 98 49 51 6C 01 00 00 00 18 48 51 6C 01 00 00 00
  00000260: 98 39 51 6C 01 00 00 00 18 38 51 6C 01 00 00 00
  00000270: 98 20 51 6C 01 00 00 00 18 28 51 6C 01 00 00 00
  00000280: 18 25 51 6C 01 00 00 00 98 25 51 6C 01 00 00 00
  00000290: 18 1C 51 6C 01 00 00 00 98 18 51 6C 01 00 00 00
  000002A0: 98 0B 51 6C 01 00 00 00 98 0C 51 6C 01 00 00 00
  000002B0: 18 04 51 6C 01 00 00 00 98 04 51 6C 01 00 00 00
  000002C0: 18 FC 50 6C 01 00 00 00 18 EF 50 6C 01 00 00 00
  000002D0: 18 EE 50 6C 01 00 00 00 98 D6 50 6C 01 00 00 00
  000002E0: 98 C3 50 6C 01 00 00 00 98 C8 50 6C 01 00 00 00
  000002F0: 98 BB 50 6C 01 00 00 00 98 B0 50 6C 01 00 00 00
  00000300: 18 B7 50 6C 01 00 00 00 98 8F 50 6C 01 00 00 00
  00000310: 18 81 50 6C 01 00 00 00 98 85 50 6C 01 00 00 00
  00000320: 98 7E 50 6C 01 00 00 00 98 83 50 6C 01 00 00 00
  00000330: 18 75 50 6C 01 00 00 00 18 5A 50 6C 01 00 00 00
  00000340: 98 7E EA 6A 01 00 00 00 18 77 EA 6A 01 00 00 00
  00000350: 18 48 F2 6A 01 00 00 00 98 6B EA 6A 01 00 00 00
  00000360: 98 8F EE 6A 01 00 00 00 18 8D EE 6A 01 00 00 00
  00000370: 18 81 EE 6A 01 00 00 00 18 88 EE 6A 01 00 00 00
  00000380: 18 89 EE 6A 01 00 00 00 98 87 EE 6A 01 00 00 00
  00000390: 18 87 EE 6A 01 00 00 00 98 85 EE 6A 01 00 00 00
  000003A0: 18 83 EE 6A 01 00 00 00 18 70 EE 6A 01 00 00 00
  000003B0: 18 7B EE 6A 01 00 00 00 98 7B EE 6A 01 00 00 00
  000003C0: 98 7C EE 6A 01 00 00 00 18 7A EE 6A 01 00 00 00
  000003D0: 98 79 EE 6A 01 00 00 00 18 79 EE 6A 01 00 00 00
  000003E0: 98 75 EE 6A 01 00 00 00 98 71 EE 6A 01 00 00 00
  000003F0: 18 72 EE 6A 01 00 00 00 18 73 EE 6A 01 00 00 00
  00000400: 18 6F EE 6A 01 00 00 00 18 6E EE 6A 01 00 00 00
  00000410: 98 6D EE 6A 01 00 00 00 18 6D EE 6A 01 00 00 00
  00000420: 98 6A EE 6A 01 00 00 00 18 68 EE 6A 01 00 00 00
  00000430: 18 61 EE 6A 01 00 00 00 18 66 EE 6A 01 00 00 00
  00000440: 98 64 EE 6A 01 00 00 00 18 64 EE 6A 01 00 00 00
  00000450: 98 63 EE 6A 01 00 00 00 98 5F EE 6A 01 00 00 00
  00000460: 98 5E EE 6A 01 00 00 00 18 5C EE 6A 01 00 00 00
  00000470: 98 50 EE 6A 01 00 00 00 98 57 EE 6A 01 00 00 00
  00000480: 98 58 EE 6A 01 00 00 00 18 57 EE 6A 01 00 00 00
  00000490: 98 56 EE 6A 01 00 00 00 18 55 EE 6A 01 00 00 00
  000004A0: 98 52 EE 6A 01 00 00 00 18 40 EE 6A 01 00 00 00
  000004B0: 18 4B EE 6A 01 00 00 00 98 4B EE 6A 01 00 00 00
  000004C0: 98 4C EE 6A 01 00 00 00 18 4A EE 6A 01 00 00 00
  000004D0: 98 49 EE 6A 01 00 00 00 18 49 EE 6A 01 00 00 00
  000004E0: 98 45 EE 6A 01 00 00 00 98 41 EE 6A 01 00 00 00
  000004F0: 18 42 EE 6A 01 00 00 00 18 43 EE 6A 01 00 00 00
  00000500: 18 3F EE 6A 01 00 00 00 18 3E EE 6A 01 00 00 00
  00000510: 98 3D EE 6A 01 00 00 00 18 3D EE 6A 01 00 00 00
  00000520: 98 3A EE 6A 01 00 00 00 18 38 EE 6A 01 00 00 00
  00000530: 18 31 EE 6A 01 00 00 00 18 36 EE 6A 01 00 00 00
  00000540: 98 34 EE 6A 01 00 00 00 18 34 EE 6A 01 00 00 00
  00000550: 98 33 EE 6A 01 00 00 00 98 2F EE 6A 01 00 00 00
  00000560: 98 2E EE 6A 01 00 00 00 18 2C EE 6A 01 00 00 00
  00000570: 98 20 EE 6A 01 00 00 00 98 27 EE 6A 01 00 00 00
  00000580: 98 28 EE 6A 01 00 00 00 18 27 EE 6A 01 00 00 00
  00000590: 98 26 EE 6A 01 00 00 00 18 25 EE 6A 01 00 00 00
  000005A0: 98 22 EE 6A 01 00 00 00 18 10 EE 6A 01 00 00 00
  000005B0: 18 1B EE 6A 01 00 00 00 98 1B EE 6A 01 00 00 00
  000005C0: 98 1C EE 6A 01 00 00 00 18 1A EE 6A 01 00 00 00
  000005D0: 98 19 EE 6A 01 00 00 00 18 19 EE 6A 01 00 00 00
  000005E0: 98 15 EE 6A 01 00 00 00 98 11 EE 6A 01 00 00 00
  000005F0: 18 12 EE 6A 01 00 00 00 18 13 EE 6A 01 00 00 00
  00000600: 18 0F EE 6A 01 00 00 00 18 0E EE 6A 01 00 00 00
  00000610: 98 0D EE 6A 01 00 00 00 18 0D EE 6A 01 00 00 00
  00000620: 98 0A EE 6A 01 00 00 00 18 08 EE 6A 01 00 00 00
  00000630: 18 01 EE 6A 01 00 00 00 18 06 EE 6A 01 00 00 00
  00000640: 18 05 EE 6A 01 00 00 00 18 EF EF 6A 01 00 00 00
  00000650: 98 E8 EF 6A 01 00 00 00 18 90 F0 6A 01 00 00 00
  00000660: 18 94 F0 6A 01 00 00 00 18 4A F0 6A 01 00 00 00
  00000670: 18 3F F0 6A 01 00 00 00 18 37 F0 6A 01 00 00 00
  00000680: 18 43 F1 6A 01 00 00 00 98 2E F0 6A 01 00 00 00
  00000690: 18 2C F0 6A 01 00 00 00 18 2A F0 6A 01 00 00 00
  000006A0: 98 1E F0 6A 01 00 00 00 18 25 F0 6A 01 00 00 00
  000006B0: 98 0F F0 6A 01 00 00 00 98 1D F0 6A 01 00 00 00
  000006C0: 98 09 F0 6A 01 00 00 00 98 0A F0 6A 01 00 00 00
  000006D0: 18 CC EF 6A 01 00 00 00 18 C9 EF 6A 01 00 00 00
  000006E0: 98 BE EF 6A 01 00 00 00 18 A9 EF 6A 01 00 00 00
  000006F0: 98 98 EF 6A 01 00 00 00 18 B1 EF 6A 01 00 00 00
  00000700: 98 8E EF 6A 01 00 00 00 18 6F EF 6A 01 00 00 00
  00000710: 18 66 EF 6A 01 00 00 00 98 59 EF 6A 01 00 00 00
  00000720: 98 4A EF 6A 01 00 00 00 18 01 EF 6A 01 00 00 00
  00000730: 98 06 EF 6A 01 00 00 00 98 19 ED 6A 01 00 00 00
  00000740: 18 FC ED 6A 01 00 00 00 98 F1 ED 6A 01 00 00 00
  00000750: 18 8F ED 6A 01 00 00 00 18 9F ED 6A 01 00 00 00
  00000760: 98 99 ED 6A 01 00 00 00 18 96 ED 6A 01 00 00 00
  00000770: 18 63 ED 6A 01 00 00 00 18 57 ED 6A 01 00 00 00
  00000780: 98 4D ED 6A 01 00 00 00 18 45 ED 6A 01 00 00 00
  00000790: 98 2E ED 6A 01 00 00 00 18 09 ED 6A 01 00 00 00
  000007A0: 98 BD EC 6A 01 00 00 00 98 F6 EC 6A 01 00 00 00
  000007B0: 18 FC EC 6A 01 00 00 00 98 EB EC 6A 01 00 00 00
  000007C0: 98 A0 EC 6A 01 00 00 00 98 E8 EC 6A 01 00 00 00
  000007D0: 18 B7 EB 6A 01 00 00 00 98 86 EC 6A 01 00 00 00
  000007E0: 18 67 EC 6A 01 00 00 00 98 4A EC 6A 01 00 00 00
  000007F0: 18 45 EC 6A 01 00 00 00 18 08 EC 6A 01 00 00 00
  00000800: 98 FC EB 6A 01 00 00 00 18 F4 EB 6A 01 00 00 00
  00000810: 18 EF EB 6A 01 00 00 00 98 EE EB 6A 01 00 00 00
  00000820: 98 DA EB 6A 01 00 00 00 18 D9 EB 6A 01 00 00 00
  00000830: 98 CD EB 6A 01 00 00 00 98 C0 EB 6A 01 00 00 00
  00000840: 98 C9 EB 6A 01 00 00 00 18 C1 EB 6A 01 00 00 00
  00000850: 18 A9 EB 6A 01 00 00 00 98 9E EB 6A 01 00 00 00
  00000860: 98 8F EB 6A 01 00 00 00 98 8C EB 6A 01 00 00 00
  00000870: 18 7F EB 6A 01 00 00 00 18 77 EB 6A 01 00 00 00
  00000880: 98 66 EB 6A 01 00 00 00 18 52 EB 6A 01 00 00 00
  00000890: 98 45 EB 6A 01 00 00 00 18 14 EB 6A 01 00 00 00
  000008A0: 18 03 EB 6A 01 00 00 00 98 E0 EA 6A 01 00 00 00
  000008B0: 18 C6 EA 6A 01 00 00 00 18 B4 EA 6A 01 00 00 00
  000008C0: 18 AB EA 6A 01 00 00 00 98 9D EA 6A 01 00 00 00
  000008D0: 18 95 EA 6A 01 00 00 00 98 8A EA 6A 01 00 00 00
  000008E0: 98 5E EA 6A 01 00 00 00 18 53 EA 6A 01 00 00 00
  000008F0: 98 EC E8 6A 01 00 00 00 18 60 E9 6A 01 00 00 00
  00000900: 98 6B E9 6A 01 00 00 00 18 63 E9 6A 01 00 00 00
  00000910: 98 8B E9 6A 01 00 00 00 18 4E EA 6A 01 00 00 00
  00000920: 98 44 EA 6A 01 00 00 00 98 38 EA 6A 01 00 00 00
  00000930: 98 28 EA 6A 01 00 00 00 18 1F EA 6A 01 00 00 00
  00000940: 98 0E EA 6A 01 00 00 00 98 9D E9 6A 01 00 00 00
  00000950: 18 5D E9 6A 01 00 00 00 98 57 E9 6A 01 00 00 00
  00000960: 18 73 E9 6A 01 00 00 00 98 0E E9 6A 01 00 00 00
  00000970: 18 F8 E8 6A 01 00 00 00 98 9A DE 6A 01 00 00 00
  00000980: 98 5C DD 6A 01 00 00 00 98 55 DD 6A 01 00 00 00
  00000990: 98 4D DD 6A 01 00 00 00 18 49 DD 6A 01 00 00 00
  000009A0: 18 3B DD 6A 01 00 00 00 98 CE CD 6A 01 00 00 00
  000009B0: 98 C5 CD 6A 01 00 00 00 98 B9 CD 6A 01 00 00 00
  000009C0: 18 AF CD 6A 01 00 00 00 98 A1 CD 6A 01 00 00 00
  000009D0: 98 8A CD 6A 01 00 00 00 18 7F CD 6A 01 00 00 00
  000009E0: 98 78 CD 6A 01 00 00 00 18 DF E8 6A 01 00 00 00
  000009F0: 18 77 CD 6A 01 00 00 00 98 D5 E8 6A 01 00 00 00
  00000A00: 98 C0 E8 6A 01 00 00 00 18 C8 E8 6A 01 00 00 00
  00000A10: 98 BD E8 6A 01 00 00 00 98 9D E8 6A 01 00 00 00
  00000A20: 98 95 E8 6A 01 00 00 00 18 72 E8 6A 01 00 00 00
  00000A30: 98 C7 E5 6A 01 00 00 00 18 AB E5 6A 01 00 00 00
  00000A40: 18 82 E5 6A 01 00 00 00 98 0F E5 6A 01 00 00 00
  00000A50: 18 FB E4 6A 01 00 00 00 98 F4 51 6C 01 00 00 00
  00000A60: 98 DD E4 6A 01 00 00 00 18 DA E4 6A 01 00 00 00
  00000A70: 18 C0 E4 6A 01 00 00 00 98 BD E4 6A 01 00 00 00
  00000A80: 18 9D E4 6A 01 00 00 00 18 5F E4 6A 01 00 00 00
  00000A90: 98 6D E4 6A 01 00 00 00 98 7E E4 6A 01 00 00 00
  00000AA0: 98 5D E4 6A 01 00 00 00 18 6C E4 6A 01 00 00 00
  00000AB0: 18 52 E4 6A 01 00 00 00 98 AC E3 6A 01 00 00 00
  00000AC0: 18 AE E2 6A 01 00 00 00 18 A3 E2 6A 01 00 00 00
  00000AD0: 18 A9 E2 6A 01 00 00 00 98 9C E2 6A 01 00 00 00
  00000AE0: 98 99 E2 6A 01 00 00 00 98 8B E2 6A 01 00 00 00
  00000AF0: 98 87 E2 6A 01 00 00 00 98 71 E2 6A 01 00 00 00
  00000B00: 98 78 E2 6A 01 00 00 00 18 6A E2 6A 01 00 00 00
  00000B10: 98 66 E2 6A 01 00 00 00 98 5F E2 6A 01 00 00 00
  00000B20: 98 58 E2 6A 01 00 00 00 98 4B E2 6A 01 00 00 00
  00000B30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  00000B40: 48 0B 00 00 00 00 00 00
• Memory dump from the 88th reboot (second ASSERT occurrence)
  00000000: 70 68 64 30 B0 4D 99 91 04 00 00 00 77 7E 98 58
  00000010: 48 0B 00 00 00 00 00 00 98 FF 51 6C 01 00 00 00
  00000020: 18 EF 51 6C 01 00 00 00 98 EC 51 6C 01 00 00 00
  00000030: 18 1F 50 6C 01 00 00 00 18 1A 50 6C 01 00 00 00
  00000040: 18 9E 4F 6C 01 00 00 00 18 14 F5 6A 01 00 00 00
  00000050: 18 FE F3 6A 01 00 00 00 18 EF F3 6A 01 00 00 00
  00000060: 18 E9 F3 6A 01 00 00 00 98 BF F3 6A 01 00 00 00
  00000070: 98 B9 F3 6A 01 00 00 00 18 50 F3 6A 01 00 00 00
  00000080: 18 59 F3 6A 01 00 00 00 18 2E F3 6A 01 00 00 00
  00000090: 98 43 F3 6A 01 00 00 00 98 48 F3 6A 01 00 00 00
  000000A0: 98 3B F3 6A 01 00 00 00 18 23 F3 6A 01 00 00 00
  000000B0: 98 10 F3 6A 01 00 00 00 18 12 F3 6A 01 00 00 00
  000000C0: 98 2C F2 6A 01 00 00 00 18 26 F2 6A 01 00 00 00
  000000D0: 18 28 F2 6A 01 00 00 00 18 08 F3 6A 01 00 00 00
  000000E0: 98 EE F2 6A 01 00 00 00 98 E6 F2 6A 01 00 00 00
  000000F0: 18 E6 F2 6A 01 00 00 00 98 DE F2 6A 01 00 00 00
  00000100: 18 9F F2 6A 01 00 00 00 18 9C F2 6A 01 00 00 00
  00000110: 18 99 F2 6A 01 00 00 00 98 93 F2 6A 01 00 00 00
  00000120: 98 7D F2 6A 01 00 00 00 98 78 F2 6A 01 00 00 00
  00000130: 18 73 F2 6A 01 00 00 00 18 63 F2 6A 01 00 00 00
  00000140: 98 5B F2 6A 01 00 00 00 98 5C F2 6A 01 00 00 00
  00000150: 98 59 F2 6A 01 00 00 00 18 4C F2 6A 01 00 00 00
  00000160: 98 47 F2 6A 01 00 00 00 18 3E F2 6A 01 00 00 00
  00000170: 18 38 F2 6A 01 00 00 00 98 35 F2 6A 01 00 00 00
  00000180: 98 15 F2 6A 01 00 00 00 98 69 F1 6A 01 00 00 00
  00000190: 98 67 F1 6A 01 00 00 00 18 5A F1 6A 01 00 00 00
  000001A0: 98 54 F1 6A 01 00 00 00 18 0C F2 6A 01 00 00 00
  000001B0: 98 0E F2 6A 01 00 00 00 18 FC F1 6A 01 00 00 00
  000001C0: 98 F2 F1 6A 01 00 00 00 18 D8 F1 6A 01 00 00 00
  000001D0: 98 AE F1 6A 01 00 00 00 18 A7 F1 6A 01 00 00 00
  000001E0: 98 A1 F1 6A 01 00 00 00 18 98 F1 6A 01 00 00 00
  000001F0: 98 83 F1 6A 01 00 00 00 18 7E F1 6A 01 00 00 00
  00000200: 98 77 F1 6A 01 00 00 00 98 72 F1 6A 01 00 00 00
  00000210: 18 AA F0 6A 01 00 00 00 18 A4 F0 6A 01 00 00 00
  00000220: 98 A8 F0 6A 01 00 00 00 18 4B F1 6A 01 00 00 00
  00000230: 98 3E F1 6A 01 00 00 00 18 48 F1 6A 01 00 00 00
  00000240: 18 39 F1 6A 01 00 00 00 18 35 F1 6A 01 00 00 00
  00000250: 98 49 51 6C 01 00 00 00 18 48 51 6C 01 00 00 00
  00000260: 98 39 51 6C 01 00 00 00 18 38 51 6C 01 00 00 00
  00000270: 98 20 51 6C 01 00 00 00 18 28 51 6C 01 00 00 00
  00000280: 18 25 51 6C 01 00 00 00 98 25 51 6C 01 00 00 00
  00000290: 18 1C 51 6C 01 00 00 00 98 18 51 6C 01 00 00 00
  000002A0: 98 0B 51 6C 01 00 00 00 98 0C 51 6C 01 00 00 00
  000002B0: 18 04 51 6C 01 00 00 00 98 04 51 6C 01 00 00 00
  000002C0: 18 FC 50 6C 01 00 00 00 18 EF 50 6C 01 00 00 00
  000002D0: 18 EE 50 6C 01 00 00 00 98 D6 50 6C 01 00 00 00
  000002E0: 98 C3 50 6C 01 00 00 00 98 C8 50 6C 01 00 00 00
  000002F0: 98 BB 50 6C 01 00 00 00 98 B0 50 6C 01 00 00 00
  00000300: 18 B7 50 6C 01 00 00 00 98 8F 50 6C 01 00 00 00
  00000310: 18 81 50 6C 01 00 00 00 98 85 50 6C 01 00 00 00
  00000320: 98 7E 50 6C 01 00 00 00 98 83 50 6C 01 00 00 00
  00000330: 18 75 50 6C 01 00 00 00 18 5A 50 6C 01 00 00 00
  00000340: 98 7E EA 6A 01 00 00 00 18 77 EA 6A 01 00 00 00
  00000350: 18 48 F2 6A 01 00 00 00 98 6B EA 6A 01 00 00 00
  00000360: 98 8F EE 6A 01 00 00 00 18 8D EE 6A 01 00 00 00
  00000370: 18 81 EE 6A 01 00 00 00 18 88 EE 6A 01 00 00 00
  00000380: 18 89 EE 6A 01 00 00 00 98 87 EE 6A 01 00 00 00
  00000390: 18 87 EE 6A 01 00 00 00 98 85 EE 6A 01 00 00 00
  000003A0: 18 83 EE 6A 01 00 00 00 18 70 EE 6A 01 00 00 00
  000003B0: 18 7B EE 6A 01 00 00 00 98 7B EE 6A 01 00 00 00
  000003C0: 98 7C EE 6A 01 00 00 00 18 7A EE 6A 01 00 00 00
  000003D0: 98 79 EE 6A 01 00 00 00 18 79 EE 6A 01 00 00 00
  000003E0: 98 75 EE 6A 01 00 00 00 98 71 EE 6A 01 00 00 00
  000003F0: 18 72 EE 6A 01 00 00 00 18 73 EE 6A 01 00 00 00
  00000400: 18 6F EE 6A 01 00 00 00 18 6E EE 6A 01 00 00 00
  00000410: 98 6D EE 6A 01 00 00 00 18 6D EE 6A 01 00 00 00
  00000420: 98 6A EE 6A 01 00 00 00 18 68 EE 6A 01 00 00 00
  00000430: 18 61 EE 6A 01 00 00 00 18 66 EE 6A 01 00 00 00
  00000440: 98 64 EE 6A 01 00 00 00 18 64 EE 6A 01 00 00 00
  00000450: 98 63 EE 6A 01 00 00 00 98 5F EE 6A 01 00 00 00
  00000460: 98 5E EE 6A 01 00 00 00 18 5C EE 6A 01 00 00 00
  00000470: 98 50 EE 6A 01 00 00 00 98 57 EE 6A 01 00 00 00
  00000480: 98 58 EE 6A 01 00 00 00 18 57 EE 6A 01 00 00 00
  00000490: 98 56 EE 6A 01 00 00 00 18 55 EE 6A 01 00 00 00
  000004A0: 98 52 EE 6A 01 00 00 00 18 40 EE 6A 01 00 00 00
  000004B0: 18 4B EE 6A 01 00 00 00 98 4B EE 6A 01 00 00 00
  000004C0: 98 4C EE 6A 01 00 00 00 18 4A EE 6A 01 00 00 00
  000004D0: 98 49 EE 6A 01 00 00 00 18 49 EE 6A 01 00 00 00
  000004E0: 98 45 EE 6A 01 00 00 00 98 41 EE 6A 01 00 00 00
  000004F0: 18 42 EE 6A 01 00 00 00 18 43 EE 6A 01 00 00 00
  00000500: 18 3F EE 6A 01 00 00 00 18 3E EE 6A 01 00 00 00
  00000510: 98 3D EE 6A 01 00 00 00 18 3D EE 6A 01 00 00 00
  00000520: 98 3A EE 6A 01 00 00 00 18 38 EE 6A 01 00 00 00
  00000530: 18 31 EE 6A 01 00 00 00 18 36 EE 6A 01 00 00 00
  00000540: 98 34 EE 6A 01 00 00 00 18 34 EE 6A 01 00 00 00
  00000550: 98 33 EE 6A 01 00 00 00 98 2F EE 6A 01 00 00 00
  00000560: 98 2E EE 6A 01 00 00 00 18 2C EE 6A 01 00 00 00
  00000570: 98 20 EE 6A 01 00 00 00 98 27 EE 6A 01 00 00 00
  00000580: 98 28 EE 6A 01 00 00 00 18 27 EE 6A 01 00 00 00
  00000590: 98 26 EE 6A 01 00 00 00 18 25 EE 6A 01 00 00 00
  000005A0: 98 22 EE 6A 01 00 00 00 18 10 EE 6A 01 00 00 00
  000005B0: 18 1B EE 6A 01 00 00 00 98 1B EE 6A 01 00 00 00
  000005C0: 98 1C EE 6A 01 00 00 00 18 1A EE 6A 01 00 00 00
  000005D0: 98 19 EE 6A 01 00 00 00 18 19 EE 6A 01 00 00 00
  000005E0: 98 15 EE 6A 01 00 00 00 98 11 EE 6A 01 00 00 00
  000005F0: 18 12 EE 6A 01 00 00 00 18 13 EE 6A 01 00 00 00
  00000600: 18 0F EE 6A 01 00 00 00 18 0E EE 6A 01 00 00 00
  00000610: 98 0D EE 6A 01 00 00 00 18 0D EE 6A 01 00 00 00
  00000620: 98 0A EE 6A 01 00 00 00 18 08 EE 6A 01 00 00 00
  00000630: 18 01 EE 6A 01 00 00 00 18 06 EE 6A 01 00 00 00
  00000640: 18 05 EE 6A 01 00 00 00 18 EF EF 6A 01 00 00 00
  00000650: 98 E8 EF 6A 01 00 00 00 18 90 F0 6A 01 00 00 00
  00000660: 18 94 F0 6A 01 00 00 00 18 4A F0 6A 01 00 00 00
  00000670: 18 3F F0 6A 01 00 00 00 18 37 F0 6A 01 00 00 00
  00000680: 18 43 F1 6A 01 00 00 00 98 2E F0 6A 01 00 00 00
  00000690: 18 2C F0 6A 01 00 00 00 18 2A F0 6A 01 00 00 00
  000006A0: 98 1E F0 6A 01 00 00 00 18 25 F0 6A 01 00 00 00
  000006B0: 98 0F F0 6A 01 00 00 00 98 1D F0 6A 01 00 00 00
  000006C0: 98 09 F0 6A 01 00 00 00 98 0A F0 6A 01 00 00 00
  000006D0: 18 CC EF 6A 01 00 00 00 18 C9 EF 6A 01 00 00 00
  000006E0: 98 BE EF 6A 01 00 00 00 18 A9 EF 6A 01 00 00 00
  000006F0: 98 98 EF 6A 01 00 00 00 18 B1 EF 6A 01 00 00 00
  00000700: 98 8E EF 6A 01 00 00 00 18 6F EF 6A 01 00 00 00
  00000710: 18 66 EF 6A 01 00 00 00 98 59 EF 6A 01 00 00 00
  00000720: 98 4A EF 6A 01 00 00 00 18 01 EF 6A 01 00 00 00
  00000730: 98 06 EF 6A 01 00 00 00 98 19 ED 6A 01 00 00 00
  00000740: 18 FC ED 6A 01 00 00 00 98 F1 ED 6A 01 00 00 00
  00000750: 18 8F ED 6A 01 00 00 00 18 9F ED 6A 01 00 00 00
  00000760: 98 99 ED 6A 01 00 00 00 18 96 ED 6A 01 00 00 00
  00000770: 18 63 ED 6A 01 00 00 00 18 57 ED 6A 01 00 00 00
  00000780: 98 4D ED 6A 01 00 00 00 18 45 ED 6A 01 00 00 00
  00000790: 98 2E ED 6A 01 00 00 00 18 09 ED 6A 01 00 00 00
  000007A0: 98 BD EC 6A 01 00 00 00 98 F6 EC 6A 01 00 00 00
  000007B0: 18 FC EC 6A 01 00 00 00 98 EB EC 6A 01 00 00 00
  000007C0: 98 A0 EC 6A 01 00 00 00 98 E8 EC 6A 01 00 00 00
  000007D0: 18 B7 EB 6A 01 00 00 00 98 86 EC 6A 01 00 00 00
  000007E0: 18 67 EC 6A 01 00 00 00 98 4A EC 6A 01 00 00 00
  000007F0: 18 45 EC 6A 01 00 00 00 18 08 EC 6A 01 00 00 00
  00000800: 98 FC EB 6A 01 00 00 00 18 F4 EB 6A 01 00 00 00
  00000810: 18 EF EB 6A 01 00 00 00 98 EE EB 6A 01 00 00 00
  00000820: 98 DA EB 6A 01 00 00 00 18 D9 EB 6A 01 00 00 00
  00000830: 98 CD EB 6A 01 00 00 00 98 C0 EB 6A 01 00 00 00
  00000840: 98 C9 EB 6A 01 00 00 00 18 C1 EB 6A 01 00 00 00
  00000850: 18 A9 EB 6A 01 00 00 00 98 9E EB 6A 01 00 00 00
  00000860: 98 8F EB 6A 01 00 00 00 98 8C EB 6A 01 00 00 00
  00000870: 18 7F EB 6A 01 00 00 00 18 77 EB 6A 01 00 00 00
  00000880: 98 66 EB 6A 01 00 00 00 18 52 EB 6A 01 00 00 00
  00000890: 98 45 EB 6A 01 00 00 00 18 14 EB 6A 01 00 00 00
  000008A0: 18 03 EB 6A 01 00 00 00 98 E0 EA 6A 01 00 00 00
  000008B0: 18 C6 EA 6A 01 00 00 00 18 B4 EA 6A 01 00 00 00
  000008C0: 18 AB EA 6A 01 00 00 00 98 9D EA 6A 01 00 00 00
  000008D0: 18 95 EA 6A 01 00 00 00 98 8A EA 6A 01 00 00 00
  000008E0: 98 5E EA 6A 01 00 00 00 18 53 EA 6A 01 00 00 00
  000008F0: 98 EC E8 6A 01 00 00 00 18 60 E9 6A 01 00 00 00
  00000900: 98 6B E9 6A 01 00 00 00 18 63 E9 6A 01 00 00 00
  00000910: 98 8B E9 6A 01 00 00 00 18 4E EA 6A 01 00 00 00
  00000920: 98 44 EA 6A 01 00 00 00 98 38 EA 6A 01 00 00 00
  00000930: 98 28 EA 6A 01 00 00 00 18 1F EA 6A 01 00 00 00
  00000940: 98 0E EA 6A 01 00 00 00 98 9D E9 6A 01 00 00 00
  00000950: 18 5D E9 6A 01 00 00 00 98 57 E9 6A 01 00 00 00
  00000960: 18 73 E9 6A 01 00 00 00 98 0E E9 6A 01 00 00 00
  00000970: 18 F8 E8 6A 01 00 00 00 98 9A DE 6A 01 00 00 00
  00000980: 98 5C DD 6A 01 00 00 00 98 55 DD 6A 01 00 00 00
  00000990: 98 4D DD 6A 01 00 00 00 18 49 DD 6A 01 00 00 00
  000009A0: 18 3B DD 6A 01 00 00 00 98 CE CD 6A 01 00 00 00
  000009B0: 98 C5 CD 6A 01 00 00 00 98 B9 CD 6A 01 00 00 00
  000009C0: 18 AF CD 6A 01 00 00 00 98 A1 CD 6A 01 00 00 00
  000009D0: 98 8A CD 6A 01 00 00 00 18 7F CD 6A 01 00 00 00
  000009E0: 98 78 CD 6A 01 00 00 00 18 DF E8 6A 01 00 00 00
  000009F0: 18 77 CD 6A 01 00 00 00 98 D5 E8 6A 01 00 00 00
  00000A00: 98 C0 E8 6A 01 00 00 00 18 C8 E8 6A 01 00 00 00
  00000A10: 98 BD E8 6A 01 00 00 00 98 9D E8 6A 01 00 00 00
  00000A20: 98 95 E8 6A 01 00 00 00 18 72 E8 6A 01 00 00 00
  00000A30: 98 C7 E5 6A 01 00 00 00 18 AB E5 6A 01 00 00 00
  00000A40: 18 82 E5 6A 01 00 00 00 98 0F E5 6A 01 00 00 00
  00000A50: 18 FB E4 6A 01 00 00 00 98 F4 51 6C 01 00 00 00
  00000A60: 98 DD E4 6A 01 00 00 00 18 DA E4 6A 01 00 00 00
  00000A70: 18 C0 E4 6A 01 00 00 00 98 AE E4 6A 01 00 00 00
  00000A80: 98 9D E4 6A 01 00 00 00 18 5F E4 6A 01 00 00 00
  00000A90: 98 6D E4 6A 01 00 00 00 98 7E E4 6A 01 00 00 00
  00000AA0: 98 5D E4 6A 01 00 00 00 18 6C E4 6A 01 00 00 00
  00000AB0: 18 52 E4 6A 01 00 00 00 98 AC E3 6A 01 00 00 00
  00000AC0: 18 AE E2 6A 01 00 00 00 18 A3 E2 6A 01 00 00 00
  00000AD0: 18 A9 E2 6A 01 00 00 00 98 9C E2 6A 01 00 00 00
  00000AE0: 98 99 E2 6A 01 00 00 00 98 8B E2 6A 01 00 00 00
  00000AF0: 98 87 E2 6A 01 00 00 00 98 71 E2 6A 01 00 00 00
  00000B00: 98 78 E2 6A 01 00 00 00 18 6A E2 6A 01 00 00 00
  00000B10: 98 66 E2 6A 01 00 00 00 98 5F E2 6A 01 00 00 00
  00000B20: 98 58 E2 6A 01 00 00 00 98 4B E2 6A 01 00 00 00
  00000B30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  00000B40: 48 0B 00 00 00 00 00 00