Skip to content

Bulk uploading: COPY ... FROM STDIN WITH CSV #11

@btakita

Description

@btakita

Is your feature request related to a problem? Please describe.
Bulk updates are significantly faster using COPY instead of single sql queries.

Describe the solution you'd like

const { stream } = pg.query(` COPY my_table (col1, col2, etc) FROM STDIN WITH CSV`, { stream: true })
createReadStream('./data.csv').on('data', $=>stream.write($)).on('close', ()=>stream.close())

Describe alternatives you've considered

  • Using psql from shell or another library/language
  • Not using COPY. Use order of magnitude slower queries instead.

Additional context
https://www.citusdata.com/blog/2017/11/08/faster-bulk-loading-in-postgresql-with-copy/

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions