Redshift Compression Macro throwing delimiter error

[ryansalazar]

Describe the bug

Upon trying to utilize the compression macro I am getting a delimiter error after a run on through CI

Steps to reproduce

This should be reproduce-able by trying to run the compression macro. I have not primary key set here

Expected results

DBT run is not able to successfully - ideally I would get a successful run and see compressed results from table output

Actual results

Error message below

System information

packages:

• package: fishtown-analytics/dbt_utils
  version: 0.5.0
• package: fishtown-analytics/redshift
  version: 0.4.1

Which database are you using dbt with?

• postgres
• redshift
• bigquery
• snowflake
• other (specify: ____________)

0.17.0

[drewbanin]

hey @ryansalazar - can you share some more information about how you are running the compression macro? Are you executing it via an operation? Or in a project/model hook? What does the code / command look like?

From the screenshot you shared, it looks like dbt is compiling SQL that includes a reference to a table called:

"<dbt.clients.jinja.MacroGenerator object at 0x7f5e10f1a630>}".""

which is obviously not going to be a real object in the database. I actually think that the } character in the schema name gives us a hint - somehow a dictionary containing the dbt context is making it's way into the "schema" field.

Let us know if you can share some more information and we'd be happy to try to help out :)

[ryansalazar]

hey Drew - yeah so I am running this at the model level. See below screenshot of code at model level

yeah so the screenshot I sent over is based on output from DBT's CI with our gitrepo (so it is a temp schema when the duplicate the repo). The model isn't passing checks and I am very hesitant to merge to master

[drewbanin]

ok - thanks for that. I think the issue comes up in the code you're using to call the compress_table macro. Your code looks like:

{{ redshift.compress_table(customer_analytics, fct_contacts_attribution, drop_backup=False) }}

Can you try quoting the customer_analytics and fct_contacts_attribution keywords? That will cause Jinja/dbt to treat them as string literals instead of variables. With the code you've shown above, they are both treated as undefined variables which is leading to the wonky behavior you're seeing. Something like this should work:

{{ redshift.compress_table("customer_analytics", "fct_contacts_attribution", drop_backup=False) }}

FWIW, dbt used to catch these undefined variables and throw a warning/error, but there were some Jinja issues that made us walk that change back. Might be worth reinvestigating it to make issues like this one easier to debug in the future!