Dynamodb Not Receiving The Entire Sqs Message Body

I am pulling data from an API in batches and sending it to an SQS Queue. Where I am having an issue is processing the message in order to send the data to DynamoDB. There is suppos

Solution 1:

Lambda Event Source Mapping for SQS will poll for messages and call Lambda function for a batch of records based on batch size which by default is 10. And processing the batch should be done by looping event["Records"] array.

Key factors that should be considered for setting batch size.

• If lambda processing fails, entire batch will be resend and will be retried by AWS. If function can't accept processing duplicate records, batchsize should be set to 1.
• If processing a single record in lambda takes 20 ms, we will still be charged for 100ms(this is minimum) by AWS, we can easily reduce 5x cost by simply setting batch size of 5.

Always recommended to

• Set a batch size higher and code lambda to be idempotent.
• Code Lambda to process all records irrespective what batch size is.