Skip to main content

Aws DynamoDb Part 2

Partitions and Performance

For a single partition:

- For a single partition key
- You can have a hard limit around 3000RCU (Read Capcity Units)
- You can have hard limit around 1000WCR
- Partition key should evenly distributed among partitions
- Accessing frequently a partition is called "hot partition" can cause performance issues.
- General number of requests you make to dynamodb too high

Partitions Best Practice

- Use high cardinality attributions - itemId, cardId, SessionId
- Use composite attributed - ComputerId+DisplayId
- Cache - can use DAX (dynamodb accelerator for reads)
- Add Random Numbers to make partition key high cardinality

Consistency Models

- DynamoDb supports both eventual and strongly consistent.
- By default eventual consistency.

- Eventual - Read response can be non latest, other write, stale, repeat get different answer, but keep repeating and will get latest answer.
- Strongly - Always up to date data, can get error 500, higher latency, not supported on global secondary index GSI, use more RCU throughput, use it when using api for GetItem then pass parameter you want ConsistentRead.


- Supports Transactions!
- All or nothing!
- ACID - Atomicity, Consistency, Isolation, Durability
- Read and write multiple items accross multiple tables
- Check pre requisite conditions before writing
- Group (Put, Update, Delete, ConditionCheck) in a single transaction
- API TransactWriteItems operation - succeed/fail
- TransactGetItems
- Cost - no additional to enable but if you have multiple reads costs.
- Like 2 phase commit - dynamo will perform 2 underlying reads or writes every time in transaction 1. prepare 2. commit.
- When we look at cloud watch we would see these two read/writes.

Scan / Query

- Scan - yeah full scan - avoid using it
- Can cause lot of RCU
- ProjectionExpression to select specific attributes (columns)
- Parallel scan for higher performance
- Set ConsistenRead on scan to get it to strong consistent

- Query based on primary key which is distinct for each row.
- like UserId
- Can use optional sort key to filter items based on sort key
- Results sorted by sort key
- ScanIndexForward for query with it can reverse results
- More efficient scan


LSI - Local Secondary Index

- Alternative sort key for use in scan and query
- 5 - Up to 5 LSI per table
- Sort key is one scala attribute
- Created only at table creation time
- Canno add remove modify LSI
- Same partition key as table - just a + different sort key
- Different view of the data
- UserId --> BirthDate , UserID --> (LSI) --> Height

GSI Global Secondary Index

- This is like a new table
- 1. Different partition key.  2. Different sort key.
- Create any time.
- Different partition key than orig table.
- Different sort key.
- UserId --> BirthDate, GSI: EmailId --> LoginTime
- Can specify which attributes to project to this "new table"
- Define RCW/WCU for this GSI
- Can effect performance and throttle main table when we have writes even if have enough WCU because need to reflect the new items.
- LSI not throttling.


Popular posts from this blog

Dev OnCall Patterns

Introduction Being On-Call is not easy. So does writing software. Being On-Call is not just a magic solution, anyone who has been On-Call can tell you that, it's a stressful, you could be woken up at the middle of the night, and be undress stress, there are way's to mitigate that. White having software developers as On-Calls has its benefits, in order to preserve the benefits you should take special measurements in order to mitigate the stress and lack of sleep missing work-life balance that comes along with it. Many software developers can tell you that even if they were not being contacted the thought of being available 24/7 had its toll on them. But on the contrary a software developer who is an On-Call's gains many insights into troubleshooting, responsibility and deeper understanding of the code that he and his peers wrote. Being an On-Call all has become a natural part of software development. Please note I do not call software development software engineering b

SQL Window functions (OVER, PARTITION_BY, ...)

Introduction When you run an SQL Query you select rows, but what if you want to have a summary per multiple rows, for example you want to get the top basketball for each country, in this case we don't only group by country, but we want also to get the top player for each of the country.  This means we want to group by country and then select the first player.  In standard SQL we do this with joining with same table, but we could also use partition by and windowing functions. For each row the window function is computed across the rows that fall into the same partition as the current row.  Window functions are permitted only in the  SELECT  list and the  ORDER BY  clause of the query They are forbidden elsewhere, such as in  GROUP BY ,  HAVING  and  WHERE  clauses. This is because they logically execute after the processing of those clauses Over, Partition By So in order to do a window we need this input: - How do we want to group the data which windows do we want to have? so  def c

Ry's Git Tutorial BOOK Review

Ry's Git Tutorial BOOK Review There are many good books about git but  Ry's Git Tutorial stands out as an amazing git tutorial book  I will describe below why: I want to stress out although I'm going to list below what this books talks about that is not the reason you should read this book.   The reason you should read this book is that it's crystal clear about git! it's just an amazing book, it's fun to read, and I truly get to understand git. It very clear, Ry's manages to bring us closer to git by making things really clear. Ry's makes an outstanding effort to bring us from knowing nothing about git to being an expert. The book starts with the real basics like Initializing the repository on an example website, staging files, exploring the repository. Then he moves on to  undoing changes  and he scans various ways you can undo your changes: viewing old revisions, tagging release, undo commit stages etc. Branches