Regression in 8.1.0: null values serialized as "null" strings (works in 7.3.1)

[ernestoalarcongallo]

Hi maintainers, thanks for your work on python-hcl2.

Environment:

python-hcl2: 8.1.0
Last known working version: 7.3.1

Summary:

After upgrading to 8.1.0, parsing/serialization appears to preserve quote characters in a way that changes value types. In our Terraform tfvars JSON output, null values become the string "null", which then breaks Terraform type validation.

Observed error:

Error: Region is not valid: "us-east-1"

And during Terraform plan:
Error: Invalid value for input variable
a number is required

Problematic generated values:

ecs_autoscaling_replicas: "null"
ecs_autoscaling_target_cpu_utilization: "null"
ecs_autoscaling_target_ram_utilization: "null"

Expected behavior:

These values should remain null (JSON null), not "null" (string), so Terraform can treat them as nullable numeric inputs.

Compatibility attempt:

We tried the migration-guide compatibility options:

V7_COMPAT = SerializationOptions(
strip_string_quotes=True,
explicit_blocks=False,
with_comments=False,
)

With these options, the null-to-"null" issue still occurs.

Impact:

Terraform planning fails because variables expecting numbers receive strings.

Request:

Could you please confirm whether this is a known regression in 8.1.0 and advise on a workaround? Happy to provide a minimal reproducer if needed.

Thanks a lot.

[Danori]

I'm noticing a similar sort of regression with boolean values. They are serialized as strings rather than python bool values. I was trying to find a fix for myself, and I believe its related to hcl2.rules.literal_rules.TokenRule. I applied the following locally:

from abc import ABC
from typing import Any, Tuple

from hcl2.rules.abstract import LarkRule, LarkToken
from hcl2.utils import SerializationOptions, SerializationContext, to_dollar_string


class TokenRule(LarkRule, ABC):
    """Base rule wrapping a single token child."""

    _children_layout: Tuple[LarkToken]
+   _KEYWORD_MAP = {
+       "true": True,
+       "false": False,
+       "null": None
+   }

    @property
    def token(self) -> LarkToken:
        """Return the single token child."""
        return self._children[0]

    def serialize(
        self, options=SerializationOptions(), context=SerializationContext()
    ) -> Any:
+       value = self.token.value
+       if isinstance(value, str) and value in self._KEYWORD_MAP:
+           return self._KEYWORD_MAP[value]
        """Serialize by delegating to the token's own serialization."""
        return self.token.serialize()

This logic was pulled basically directly from hcl2.query.predicate.py.

However, I am by no means a maintainer and this was a hacky local solution. Hope this helps @kkozik-amplify, thank you for maintaining this project !

[kkozik-amplify]

Hi @ernestoalarcongallo @Danori
Thank you for the detailed reports! We have a pending PR #294 and hopefully the fix can be released today in a new version 8.1.2. #294 merged and released as 8.1.2

While the fix proposed by @Danori solves the most common use case, the issue itself is broader - pre v8, true/false/null literals were handled roughly as in the above proposed solution (which even back then caused several ripple effects). Because of that, when working on v8 architectural overhaul we missed proper handling of the literals on the lark grammar level, which propagated deeper into the serialization (and deserialization) chain. PR #294 addresses the problem comprehensively.