admin 管理员组

文章数量: 1086019

A lambda that DELETEs and COPYs 8 parquet files from S3 into 8 Redshift tables (using boto3).

Each table is created from a single file, each with less than 3K rows.

7 of the queries take 2-3 seconds each, and one takes 50+ seconds.

I have exhausted my checks and not sure what this can be, this didn't work:

  • Removed all fks/constraints
  • VACUUM/ANALYZE the tables
  • Recreated the tables
  • Changed the isolation levels
  • Increased the Lambda memory/CPU allocation

Notable considerations:

  • When I run it directly on Redshift (not the datashare), the COPY takes 2s.
  • It's definitely the COPY and not the DELETE (confirmed via logging).

Any suggestions appreciated!

本文标签：

Error[2]: Invalid argument supplied for foreach(), File: /www/wwwroot/roclinux.cn/tmp/view_template_quzhiwa_htm_read.htm, Line: 58
File: /www/wwwroot/roclinux.cn/tmp/route_read.php, Line: 205, include(/www/wwwroot/roclinux.cn/tmp/view_template_quzhiwa_htm_read.htm)
File: /www/wwwroot/roclinux.cn/tmp/index.inc.php, Line: 129, include(/www/wwwroot/roclinux.cn/tmp/route_read.php)
File: /www/wwwroot/roclinux.cn/index.php, Line: 29, include(/www/wwwroot/roclinux.cn/tmp/index.inc.php)

版权声明：本文标题：aws lambda - Why does one S3-to-Redshift COPY operation take 50+ seconds while seven identical operations take 2-3 seconds? - St 内容由网友自发贡献，该文观点仅代表作者本人， 转载请联系作者并注明出处：http://roclinux.cn/p/1744073327a2528891.html， 本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌抄袭侵权/违法违规的内容，一经查实，本站将立刻删除。