I'm new to DynamoDb and trying to do bulk insert of around 5.5k items on my locally running DynamoDb using Java. Despite my best efforts, and various tweakings, I am able to do this in around 100 seconds(even after using executor framework).
I posted my code here, but didn't get an answer.
To improve the insertion rate I tried changing the Provisioned Throughput value several times while creating the table, then I got to know that when running locally, dynamodb ignores the throughput values. So I think its my dynamodb that is not able to handle so many write requests at a time and when I do it on the AWS server, the performance might improve.
This is the code I was running to create table:
public static void main(String[] args) throws Exception {
AmazonDynamoDBClient client = new AmazonDynamoDBClient().withEndpoint("http://localhost:8000");
DynamoDB dynamoDB = new DynamoDB(client);
String tableName = "Database";
try {
System.out.println("Creating the table, wait...");
Table table = dynamoDB.createTable(tableName, Arrays.asList(new KeySchemaElement("Type", KeyType.HASH),
new KeySchemaElement("ID", KeyType.RANGE)
), Arrays.asList(new AttributeDefinition("Type", ScalarAttributeType.S),
new AttributeDefinition("ID", ScalarAttributeType.S)),
new ProvisionedThroughput(10000L, 10000L));
table.waitForActive();
System.out.println("Table created successfully. Status: " + table.getDescription().getTableStatus());
} catch (Exception e) {
System.err.println("Cannot create the table: ");
System.err.println(e.getMessage());
}
}
But to be sure I want to know what's the default read and write capacity unit of a locally running dynamodb instance?
DynamoDBLocal does not implement the throughput limitations in any way. If your throughput is limited, it is limited by the hardware you are running it on.
From the DynamoDB Local docs:
The speed of read and write operations on table data is limited only by the speed of your computer.
According to this answer to a related question, the performance is noticeably poor because DynamoDB local uses SQLite behind the scenes. Since the implementation is different than the real DynamoDB, we should expect that the performance will be different as well.
If you need to do any performance testing with DynamoDB, you should use the real DynamoDB. My company has used DynamoDB for applications with tens of thousands of reads and writes per second without any scaling problems from DynamoDB, so I can attest that the real DynamoDB will perform much better than DynamoDB Local.