Why Uber Switched from PostgreSQL to MySQL
A detailed explanation of how PostgreSQL internal data representation and replication works, and why MySQL is more efficient:

- PostgreSQL uses immutable row data via "tuples" with a unique ctid (physical disk offset)
- Both primary and secondary indexes point to physical ctids, not logical rows. In MySQL, only primary indexes do this
- MySQL uses an extra layer of indirection: secondary index records point to primary index records, and the primary index itself holds the on-disk row locations. If a row offset changes, only the primary index needs to be updated
- PostgreSQL suffers of write amplification: a single update requires lots of changes, even on indexes not related to the updated column
- PostgresSQL creates a copy of updated rows with a new ctid, whereas MySQL updates data in-place and creates a rollback segment. This also impacts vacuuming
- Replication via WAL files is less efficient because of write amplification and the way changes are propagated and applied on each node
- PostgreSQL is more prone for data corruption on its WAL files
- Upgrading a PostgreSQL version is much harder
- MySQL uses a more complex but efficient caching strategy
- MySQL spawns a thread per each connection, PostgreSQL spawns a process. This seriously limits the number of supported concurrent connections
mysql  postgresql  comparison  databases  sql  storage  uber  blogs  articles  performance  scalability  clustering 
2 days ago
Build highly available MySQL applications using Amazon Aurora Multi-Master
Amazon Aurora Multi-Master is available for the MySQL-compatible edition of Aurora. Every database node in the cluster is a writer node that can run read and write statements. In the event of instance or AZ failures, Aurora Multi-Master enables the Aurora database to maintain read and write availability with zero application downtime.

Now for the catches: load-balancing between the writers must be done at the application level, with logic for retries and load distribution done by hand. Check GRAW in case you need consistent reads across all the cluster, at the expense of a slight performance penalty. Also, at the time of writing Aurora Multi-Master only supports two-node clusters in a single Region. Support for more writer nodes and placement of writers in multiple Regions is planned for future releases.
amazon  aws  mysql  sql  databases  aurora  scalability  reference  articles  blogs  availability  architecture  storage  clustering 
7 days ago
If you specify an ON DUPLICATE KEY UPDATE clause and a row to be inserted would cause a duplicate value in a UNIQUE index or PRIMARY KEY, an UPDATE of the old row occurs.

Compare it with REPLACE which works exactly like INSERT, except that if an old row in the table has the same value as a new row for a PRIMARY KEY or a UNIQUE index, the old row is deleted before the new row is inserted.

In some old code we were (incorrectly) using REPLACE, causing millions of newly created rows in a table. When we switched to INSERT ... ON DUPLICATE KEY UPDATE, only a few thousand rows were created.
work  amazon  mysql  aurora  sql  databases  optimization  performance  documentation  tips  reference 
8 days ago
Working with fetch size on MySQL queries
This is the documentation for the MySQL driver, scroll down to the ResultSet section. Here's where they document that the only way to fetch rows one-by-one is by setting the fetch size to Integer.MIN_VALUE. Not even setting the value to 1 will work! All this applies to jOOQ because it's using the MySQL driver under the hood.

And the weirdest part of all: this is the fastest of all fetching strategies, faster than setting the fetch size to 1, or to a bigger number, or using seek, or using limit/offset and we don't know why. The initial cost for fetching is minimal, but it'll increase when we actually consume the stream.

Notice that this behaviour applies to the MySQL driver, it might be different with the MariaDB driver (the recommended one for Aurora).
java  sql  databases  lib  jdbc  orm  performance  optimization  work  amazon  documentation  jooq  aurora  mysql 
8 days ago
INSERT ... VALUES with a single row
How to insert multiple values at the same time. Notice that using this option can give better performance than using batch operations.
jooq  java  sql  databases  lib  orm  performance  optimization  work  amazon  documentation  code  programming  howto  aurora 
10 days ago
Using JDBC batch operations
How to insert data in batches. Beware! if you're not careful you'll end up using several queries, make sure to use the single query variant. Notice that using INSERT ... VALUES with multiple rows can give better performance.
jooq  java  sql  databases  lib  orm  performance  optimization  work  amazon  documentation  code  programming  howto  aurora 
10 days ago
Serverless Computing: One Step Forward, Two Steps Back
In this paper we address critical gaps in 1st-generation serverless computing, which place its autoscaling potential at odds with dominant trends in modern computing. Some of the current shortcomings:

1. Limited Lifetimes
2. I/O Bottlenecks
3. Communication Through Slow Storage
4. No Specialized Hardware

With the natural consequences for the Function as a Service paradigm:

1. FaaS is a Data-Shipping Architecture
2. FaaS Stymies Distributed Computing
3. FaaS stymies hardware-accelerated software innovation
4. FaaS discourages Open Source service innovation

Possible improvements:

1. Fluid Code and Data Placement
2. Heterogeneous Hardware Support
3. Long-Running, Addressable Virtual Agents
4. Disorderly programming
5. Flexible Programming, Common IR
6. Service-level objectives & guarantees
7. Security concerns
cloud  aws  serverless  lambda  papers  scalability  languages  programming  pdf 
10 days ago
Limitations of Running AWS Lambda Functions Inside a VPC
If you have a need for serious concurrent executions, Lambda functions might not be the best fit. Do your best to avoid running Lambda functions inside a VPC, unless you absolutely have to. You'll run out of ENIs pretty fast, debugging will be hard and you'll start getting these errors: EC2ThrottledException, Client.NetworkInterfaceLimitExceeded, Client.RequestLimitExceeded
aws  lambda  serverless  networking  amazon  tips  articles  blogs  work  concurrency  implementation  infrastructure  debug 
12 days ago
Lazy Fetching in jOOQ
Instead of using the Seek Method (see my previous bookmark), we'll be using lazy loading to cut on memory consumption and because it's faster. I would use this when we need to read all the results. We have two options:

- fetchSize(int) + fetchLazy(), which returns a Cursor
- fetchSize(int) + fetchStreamInto(Class<E>) which returns a Stream<E>

The second is preferred when using Java 8. Either one requires that we explicitly close the underlying ResultSet using a try-with-resources.

There's a trade-off: the Seek Method uses more memory but frees resources faster and makes a single DB request per batch. On the other hand, lazy loading uses less memory but leaves an open cursor on the underlying DB, and fetches records one-by-one (if you set fetchSize to Integer.MIN_VALUE) or in small chunks. This is useful when result sets are large or if you have a lot of network latency.

VERY IMPORTANT: counter-intuitively, reading records one-by-one is faster than all the other options: cursors with a fetch size >= 1, seek, limit/offset. See the documentation on the MySQL driver.
java  sql  databases  lib  orm  performance  optimization  work  amazon  documentation  code  programming  howto  lazy  streams  jooq 
14 days ago
Finds bugs in your shell scripts. You can cabal, apt, dnf, pkg or brew install it locally right now. Looks very useful!
bash  shell  scripting  debug  tools  os  linux  commands  programming 
14 days ago
CDK All The Things: A Whirlwind Tour
A quick overview of all CDK things. Interesting points:

- CDK is written in typescript and translated to other languages via jsii
- The CDK provides two levels of constructs: Low and High, with different levels of power/control
- Not every attribute is supported at the High level, but we can always get down to the Low level or use `addPropertyOverride`
- `cdk diff` provides an extremely friendly visualization of code changes
- In the end all the code gets translated to CloudFormation (CF) templates
- We can see the generated template via `cdk synthesize`
- CDK is bound to the same limitations as CF: if CF doesn't provide a feature, CDK won't support it either
- Some CF features are not supported yet
aws  amazon  infrastructure  cloud  architecture  programming  templates  lib  java  python  documentation  work  c#  javascript  cdk  code  examples  api  cloudformation 
14 days ago
You're thinking about AWS Lambda cold starts all wrong
Cold start happens once for each concurrent execution of your function. Mitigations:

- Pre-warm your APIs by means of a scheduled cron job (CloudWatch schedule + Lambda)
- Write your Lambda functions in a language that doesn’t incur a high cold start time (Python, Go, Node.js)
- Choose a higher memory setting
- Optimize your function's dependencies and package size
- Stay as far away from VPCs as you possibly can!
aws  lambda  serverless  performance  amazon  java  jvm  tips  articles  blogs  work  concurrency  implementation  infrastructure 
17 days ago
Lambda concurrency limits and SQS triggers don't mix well
This problem, christened as "SQS overpull" means that if you set concurrency limits too low (for Lambdas consuming from a queue), a lot of your messages will end up in the DLQ. The second part of the article provides a concrete example. Mitigations:

- Set the queue’s visibility timeout to at least 6 times the timeout that you configure on your function.
- Set the maxReceiveCount on the queue’s redrive policy to at least 5.
aws  sqs  lambda  amazon  concurrency  work  articles  blogs  bugs  infrastructure  serverless  implementation 
18 days ago
AWS Cloud Development Kit (CDK) - Documentation
Documentation for the CDK. Create and provision AWS infrastructure deployments predictably and repeatedly, using your favorite programming language! Python, TypeScript, Java, C# and JavaScript currently supported.
aws  amazon  infrastructure  cloud  programming  architecture  templates  lib  java  javascript  python  documentation  work  c#  cdk 
20 days ago
AWS Cloud Development Kit (CDK) - Examples
Repo with examples for the CDK. Create and provision AWS infrastructure deployments predictably and repeatedly, using your favorite programming language! Python, TypeScript, Java, C# and JavaScript currently supported.
aws  amazon  infrastructure  cloud  programming  architecture  templates  lib  java  javascript  python  examples  work  git  c#  cdk 
20 days ago
AWS Cloud Development Kit (CDK) - Tutorial
Tutorial for the CDK. Create and provision AWS infrastructure deployments predictably and repeatedly, using your favorite programming language! Python, TypeScript, Java, C# and JavaScript currently supported.
aws  amazon  infrastructure  cloud  programming  architecture  templates  lib  java  javascript  python  c#  tutorial  work  cdk 
20 days ago
AWS Well-Architected Tool
The AWS Well-Architected Tool helps you review the state of your workloads and compares them to the latest AWS architectural best practices. The tool is based on the AWS Well-Architected Framework, developed to help cloud architects build secure, high-performing, resilient, and efficient application infrastructure.
aws  cloud  reference  architecture  design  templates  howto  implementation  work  amazon  tools 
21 days ago
Faster SQL Pagination with jOOQ Using the Seek Method
SQL LIMIT/OFFSET clauses are slow. They’re slow for a simple reason: in order to reach a high offset from a result set, all previous records have to be skipped and counted.

A much faster alternative is to use the Seek Method, which does not skip records before an OFFSET, but it skips records until the last record previously fetched. This technique is provided by the jOOQ ORM library. I would use this when we're paginating results but we do not intend to read them all.
java  sql  databases  lib  orm  performance  work  amazon  articles  blogs  code  programming  howto  optimization  jooq 
21 days ago
A modern load testing tool written in Python, define user behaviour with Python code. Locust supports running load tests distributed over multiple machines, and can therefore be used to simulate millions of simultaneous users. This is simpler to use than TPGenerator, and comes with built-in visualizations.
load  testing  python  code  work  amazon  concurrency  visualization  tools  distributed  scalability  profiling 
4 weeks ago
A hands-on session with Google Guice
Practical code example of using Guice, the dependency injection framework
injection  tutorials  java  howto  articles  code  google  lib  programming  oo  tools  framework 
7 weeks ago
Unit tests with Mockito
This tutorial explains testing with the Mockito framework for writing software tests. Mockito enables powerful (albeit verbose) testing capabilities for Java and Android code.
testing  tutorials  java  howto  articles  mocking  work  amazon  software  framework  android 
7 weeks ago
Project Lombok
Project Lombok is a Java library that automatically plugs into your editor and build tools, spicing up your Java. Never write another getter or equals method again, with one annotation your class has a fully featured builder, Automate your logging variables, and much more.
work  amazon  tools  annotations  java  lib  ide  build  metaprogramming  programming  code 
8 weeks ago
10 Best Practices for Better RESTful API
Typically we use a RESTful design for our web APIs. The concept of REST is to separate the API structure into logical resources. There are used the HTTP methods GET, DELETE, POST and PUT to operate with the resources. These are 10 best practices to design a clean RESTful API.
rest  http  api  webservices  work  amazon  design  tips  howto  web  webapps 
8 weeks ago
Dagger Dependency Injection
Building a dependency object graph via injection is important to achieve Separation of Concerns of construction and use of objects and for simplifying code testing, at the expense of separating behavior from construction. Dagger is one of the frameworks we use at work.
java  lib  oo  injection  work  tools  amazon  programming  code  google  framework 
9 weeks ago
Guice Dependency Injection
Building a dependency object graph via injection is important to achieve Separation of Concerns of construction and use of objects and for simplifying code testing, at the expense of separating behavior from construction. Guice is one of the frameworks we use at work.
java  lib  oo  injection  work  tools  amazon  programming  code  google  framework 
9 weeks ago
Aurora Serverless Data API
In order to resolve this mismatch between serverless applications and relational databases, we are launching a Data API for the MySQL-compatible version of Amazon Aurora Serverless. This API frees you from the complexity and overhead that come along with traditional connection management, and gives you the power to quickly and easily execute SQL statements that access and modify your Amazon Aurora Serverless Database instances.

The Data API is designed to meet the needs of both traditional and serverless apps. It takes care of managing and scaling long-term connections to the database and returns data in JSON form for easy parsing. All traffic runs over secure HTTPS connections.
aws  databases  aurora  serverless  amazon  api  storage  cloud  architecture  persistence 
9 weeks ago
AWS Quick Starts
Quick Starts are built by AWS solutions architects and partners to help you deploy popular technologies on AWS, based on AWS best practices for security and high availability. These accelerators reduce hundreds of manual procedures into just a few steps, so you can build your production environment quickly and start using it immediately.

Each Quick Start includes AWS CloudFormation templates that automate the deployment and a guide that discusses the architecture and provides step-by-step deployment instructions.
amazon  aws  cloud  architecture  security  availability  infrastructure 
9 weeks ago
S3 vs. DynamoDB Price Comparison
Sometimes using DynamoDB can be cheaper than S3. It all depends on the amount of information being stored and the expected read/write patterns
aws  amazon  money  comparison  cloud  databases  storage  blogs  articles  reference 
april 2019
AWS App Mesh – Application-Level Networking for Cloud Applications
AWS App Mesh is a service mesh based on the Envoy proxy (which you need to deploy and configure!) that makes it easy to monitor and control microservices. App Mesh standardizes how your microservices communicate, giving you end-to-end visibility and helping to ensure high-availability for your applications.

Interestingly, this doesn't seem to work with serverless apps, it's more targeted for EC2 and/or containerized apps. Looks useful for monitoring, logging and A/B testing microservices, also for infrastructures transitioning to containers. Consider using it in tandem with AWS X-Ray.
aws  architecture  microservices  amazon  distributed  proxy  monitoring  availability  networking  cloud  containers 
april 2019
Advanced Request Routing for AWS Application Load Balancers
Now ALBs can route requests based on host headers, path, HTTP header, HTTP request method, query string and source IP. Possible usage scenarios include A/B testing, blue/green deployments, route traffic to microservice handlers, etc. Just beware of the limits, each ALB can have up to 100 rules
aws  load  networking  amazon  http  apps  webapps  testing  blogs  reference  articles 
april 2019
AWS Toolkit for IntelliJ
The AWS Toolkit for the IntelliJ IDEA, an open source plug-in that makes it easier to create, step-through debug, build, and deploy Java applications on AWS. It provides support for serverless applications, providing local debugging via containers. Also it uses AWS SAM to define and configure resources. Support across other AWS services is planned for the future
ide  java  aws  lambda  serverless  tools  debug  deploy  opensource  containers  docker  tutorials  howto 
april 2019
Computational Imaging for VLBI Image Reconstruction
Algorithm used for imaging the first black hole, by the amazing Dr. Katherine Bouman!
papers  algorithms  visualization  space  physics  research  cs  mit 
april 2019
Los 20 clásicos del siglo XXI que hay que probar en Madrid
De los torreznos de La Raquetista a los buñuelos de Lakasa pasando por las costillas de Kitchen 154, estos son los platos y bebidas de la capital que se han convertido en leyenda en las dos últimas décadas
spain  madrid  restaurant  food  articles  fun 
march 2019
Building serverless apps with components from the AWS Serverless Application Repository
The AWS Serverless Application Repository allows developers to deploy, publish, and share common serverless components among their teams and organizations. Its public library contains components built and published using AWS SAM - the infrastructure as code, YAML language, used for templating AWS resources. The recommended approach for evaluating an AWS SAR component is:

1. Check component permissions
2. Inspect the component implementation
3. Deploy and run the component in a restricted environment
4. Monitor the component’s behavior and cost before using in production
aws  cloud  reference  architecture  design  templates  deploy  howto  implementation  serverless  lambda  amazon  articles  blogs  work  examples 
march 2019
AWS Serverless Application Repository
The AWS Serverless Application Repository is a managed repository for serverless applications. It enables teams, organizations, and individual developers to store and share reusable applications, and easily assemble and deploy serverless architectures in powerful new ways. Using the Serverless Application Repository, you don't need to clone, build, package, or publish source code to AWS before deploying it. Instead, you can use pre-built applications from the Serverless Application Repository in your serverless architectures, helping you and your teams reduce duplicated work, ensure organizational best practices, and get to market faster
aws  cloud  reference  architecture  design  templates  deploy  howto  implementation  serverless  lambda  work  amazon 
march 2019
AWS Solutions
AWS Solutions are technical reference implementations built and vetted by AWS Architects and AWS Partners, built to be operationally effective, performant, reliable, secure, cost effective, and be Well-Architected. Every AWS Solution comes with a detailed architecture diagram, a deployment guide, and instructions for both manual and automated deployment
aws  cloud  reference  architecture  security  performance  design  templates  deploy  howto  implementation  serverless  work  amazon 
march 2019
Threading Questions in Job Interviews
Some tricky interview questions about threads in Java
interview  java  reference  threads  articles  concurrency 
february 2019
Announcing WebSocket APIs in Amazon API Gateway
Starting today, you can build bidirectional communication applications using WebSocket APIs in Amazon API Gateway using a serverless architecture. This post demonstrates a simple chat using web sockets over API Gateway, with Lambdas and DynamoDB for the backend. So simple!
aws  websockets  javascript  node.js  api  networking  cloud  examples  tutorials  code  serverless  lambda  architecture  articles  reference  chat  nosql 
january 2019
API Gateway vs Application Load Balancer—Technical Details
Part one of a three-part series comparing APIGW vs. ALB. This article takes a look at the technical aspects and differences; the other two talk about the costs and provide a hands-on hello world tutorial. Curiously, the post about the costs cites one of my tweets (!)
amazon  comparison  aws  api  networking  load  architecture  articles  blogs  rest  serverless  lambda 
december 2018
AWS Serverless Application Model
The AWS Serverless Application Model (AWS SAM) is an open-source framework that enables you to build serverless applications on AWS. It provides you with a template specification (built on top of CloudFormation) to define your serverless application, and a command line interface (CLI) tool.

The Developer Guide Introduces you to AWS SAM, and shows you how to use AWS SAM to define, test, and deploy serverless applications.

The Template Specification describes the open-source specification that you use to define serverless applications.
aws  cloud  serverless  amazon  lambda  deploy  architecture  testing  opensource  shell 
november 2018
CV Compiler
CV Compiler is a new web app that uses machine learning to analyze and repair your technical resume, allowing you to shine to recruiters at Google, Yahoo and Facebook. The app essentially checks your resume and tells you what to fix and where to submit it. It's been completely bootstrapped thus far and they're working on new and improved machine learning algorithms while maintaining a library of common CV fixes
work  tools  google  yahoo  facebook  machine-learning  webapps  algorithms 
november 2018
GitLab High Availability on AWS
In this article we’ll go through a basic HA setup where we’ll start by configuring our Virtual Private Cloud and subnets to later integrate services such as RDS for our database server and ElastiCache as a Redis cluster to finally manage them within an auto scaling group with custom scaling policies.

Since the time of writing, EFS supports provisioned IOPS, making it a viable alternative to NFS. See: https://aws.amazon.com/blogs/aws/new-provisioned-throughput-for-amazon-elastic-file-system-efs
git  aws  documentation  infrastructure  howto  availability  rds  redis  networking  work  amazon 
november 2018
Why is the ibdata1 file continuously growing in MySQL?
After some research you realize that most of the disk space is used by the InnoDB’s shared tablespace ibdata1. You have innodb_file_per_table enabled, so the question is: What is stored in ibdata1? UNDO_LOGs for uncommitted transactions! and you cannot reclaim the space, this is very bad DB design!
amazon  work  databases  sql  mysql  storage  bugs  blogs  reference  transactions  relational 
october 2018
New AWS Pricing Calculator for EC2 and EBS
The new calculator is designed to help you estimate and understand your eventual AWS costs. We did our best to avoid excessive jargon and to make the calculations obvious, transparent, and accessible. You can see the options that are available to you, explore the associated costs, and make high-quality data-driven decisions
aws  tools  amazon  money  cloud  howto  blogs  reference 
october 2018
ECS Vs. EKS Vs. Fargate: The Good, the Bad, the Ugly
A great comparison of the pros and cons of ECS, EKS and Fargate. Note that ECS and EKS are alternatives to each other, whereas Fargate complements either one.
aws  containers  architecture  docker  kubernetes  fargate  comparison  blogs 
september 2018
What Do Amazon ECS Kubernetes Service and Fargate Mean for You?
In short, ECS is a simpler container alternative (Docker as a service) that integrates better with AWS.

EKS (Kubernetes as a service) is more powerful and gives you more control, but it's more complex to use. If you need even more control EKS is not for you, after all is a managed service.

Fargate simplifies management and can be used with either ECS or EKS (starting from late 2018). Think of it as an alternative to EC2. Also mind the costs, they can grow fast!
amazon  aws  containers  docker  kubernetes  fargate  blogs  articles 
september 2018
Introducing AWS Fargate – Run Containers without Managing Infrastructure
AWS Fargate is an easy way to deploy your containers on AWS. To put it simply, Fargate is like EC2 but instead of giving you a virtual machine you get a container. It’s a compute engine that allows you to use containers as a fundamental compute primitive without having to manage the underlying instances.

Is Fargate the next big thing? it simplifies the management of resources in containerized environment, instead of launching ECS tasks in EC2, use Fargate. In 2018 it'll also be available for EKS
aws  amazon  containers  docker  fargate  articles  infrastructure 
september 2018
Load Testing for Containers
Load-testing a containerized application using Scala's Gatling. Shows the basics of how to build an application inside a Docker container via a CloudFormation template and deploy it using Fargate and its auto-scaling policies, which take care of the physical part of scaling. Search more videos under the "Well-Architected" category to learn about testing Serverless applications, etc.
load  testing  containers  aws  scala  tutorials  scalability  docker  fargate  howto 
september 2018
Monitoring Amazon Aurora performance metrics
This post is part 1 of a 3-part series about monitoring Amazon Aurora database service on Amazon RDS. Part 2 is about collecting metrics from Aurora, and Part 3 details how to monitor Aurora with Datadog.

In essence, you should use CloudWatch to monitor query throughput, query latency and errors, disk queue depth and client connections and errors
amazon  work  performance  metrics  monitoring  rds  aurora  sql  databases 
september 2018
How Percona XtraBackup Works
Percona XtraBackup is based on InnoDB‘s crash-recovery functionality. It copies your InnoDB data files, which results in data that is internally inconsistent; but then it performs crash recovery on the files to make them a consistent, usable database again
mysql  clustering  databases  tools  backup  work  amazon  reference  howto 
september 2018
mysqldbcompare — Compare Two Databases and Identify Differences
This utility compares the objects and data from two databases to find differences. It identifies objects having different definitions in the two databases and presents them in a diff-style format of choice. Differences in the data are shown using a similar diff-style format
mysql  clustering  databases  tools  work  amazon  comparison 
september 2018
Percona XtraBackup
Percona XtraBackup is an open-source hot backup utility for MySQL - based servers that doesn’t lock your database during the backup. It can back up data from InnoDB, XtraDB, and MyISAM tables on MySQL 5.1 [1], 5.5, 5.6 and 5.7 servers, as well as Percona Server with XtraDB
mysql  clustering  databases  tools  backup  work  amazon 
september 2018
Backtracking an Aurora DB Cluster
With Amazon Aurora with MySQL compatibility, you can backtrack a DB cluster to a specific time, without restoring data from a backup. Backtracking "rewinds" the DB cluster to the time you specify.

Gotchas: it has to be enabled at DB creation, a rewind affects the entire cluster, is not supported with binary log (binlog) replication, is not compatible with MySQL 5.7 and causes a brief DB instance disruption
backup  recovery  databases  mysql  aws  rds  work  amazon 
september 2018
Recover from a disaster with delayed replication in Amazon RDS for MySQL
Amazon RDS for MySQL now supports a delayed replication, which allows you to set a time period that a replica database lags behind a source database. A delay is very helpful when you need to recover from a human error. For example, if someone accidentally drops a table from your primary database, that accident doesn’t have to be saved in a replica
backup  recovery  databases  mysql  aws  rds  work  amazon 
september 2018
MySQL High Availability at GitHub
How GitHub implemented HA for their MySQL databases. A multi-layer solution, segregating each responsibility in a different service: orchestration, proxy, service-discovery via orchestrator/raft, GLB/HAProxy, Consul
availability  clustering  databases  mysql  distributed  load  networking  opensource 
september 2018
Leveraging ProxySQL with AWS Aurora to Improve Performance
At the moment, the way AWS offers data connectivity with the Cluster endpoints and Reader endpoints can negatively affect performance. Also the lack of configuration and the limitation of using different endpoints lead to confusion and less optimized utilization.

But fear not, you can use ProxySQL with AWS Aurora to further leverage database performance and improve resource usage for the DB nodes.
aws  databases  sql  rds  aurora  amazon  infrastructure  work  performance  optimization  proxy  clustering 
august 2018
Amazon Aurora – Looking Deeper
There are several differences in the way Aurora works under the hood, including: bug fixes, work with transactional isolation, query cache, default config, diagnostics, and the fact that it's stuck with an old version of MySQL for licensing reasons
aws  databases  sql  rds  aurora  amazon  infrastructure  work  architecture  clustering 
august 2018
When Should I Use Amazon Aurora and When Should I use RDS MySQL?
- If you are looking for a native HA solution then you should use Aurora
- For a read-intensive workload within an HA environment, Aurora is better
- Aurora performance is great but is not as much as expected for write-intensive workloads when secondary indexes exist
- If you need to use MySQL plugins you should use RDS MySQL
- If you only need a cheaper managed solution and out of the box availability is not your main concern, RDS MySQL is better
aws  mysql  databases  sql  comparison  rds  aurora  amazon  infrastructure  work  clustering 
august 2018
Raft - Understandable Distributed Consensus
A visualization of how the Raft protocol implements a distributed consensus.
distributed  protocol  visualization  reference  networking  clustering 
august 2018
Iron Maiden Concert Setlist at Wanda Metropolitano, Madrid on July 14, 2018
Iron Maiden's setlist for the Legacy of the Beast Tour in Madrid, 14/07/2018
music  concerts  madrid  spain  setlist  metal  ironmaiden 
july 2018
Amazon Aurora Migration Handbook
This paper examines the following major contributors to the success of every database migration project:

- Factors that justify the migration to Amazon Aurora, such as compatibility, performance, cost, and high availability and durability
- Best practices for choosing the optimal migration method
- Best practices for planning and executing a migration
- Migration troubleshooting hints
amazon  databases  rds  aurora  aws  mysql  migration  sql  documentation  reference  howto  pdf  work  papers 
july 2018
AWS Application Auto Scaling
Similar to EC2 Auto Scaling, Application Auto Scaling allows you to automatically scale compute and data resources such as Amazon DynamoDB, Amazon ECS, Amazon RDS Aurora replicas, Amazon Appstream 2.0, Amazon SageMaker, Amazon EC2 Spot Fleet, or Amazon EMR. Now, any service that you build with adjustable resource capacity can be automatically scaled using the new Custom Resource Scaling feature of Application Auto Scaling
aws  scalability  apps  amazon  work  reference  documentation  howto 
july 2018
Puppet vs. Chef vs. Ansible vs. SaltStack
Puppet, Chef, Ansible and SaltStack present different paths to achieve a common goal of managing large-scale server infrastructure efficiently, with minimal input from developers and sysadmins. All four configuration management tools are designed to reduce the complexity of configuring distributed infrastructure resources, enabling speed, and ensuring reliability and compliance. This article explores the mechanism, value propositions and concerns pertaining to each configuration management solution
python  ruby  tools  devops  management  servers  comparison  articles  infrastructure 
july 2018
About Pool Sizing
Configuring a connection pool is something that developers often get wrong. There are several, possibly counter-intuitive for some, principles that need to be understood when configuring the pool. Bottomline: the smaller your pool, the better. And use this formula: connections = ((core_count * 2) + effective_spindle_count)
oracle  databases  profiling  performance  threads  optimization  pool 
july 2018
Recovering MySQL replication after error 1236
In other words, the replica is requesting data at a certain point in the log (its current position), but the master’s log file doesn’t reach that point (so there are missing entries). I've seen this error in the field, quite scary little thing.
mysql  clustering  databases  bugs  logs  transactions  work  amazon 
july 2018
Los 10 mejores restaurantes del mundo
Según "The World’s 50 Best Restaurants". Hay 3 en España, hay que ir!
food  fun  restaurant  spain 
july 2018
Cecilia en Suecia
Estudiante del Doctorado en Ciencias de la Electrónica de Unicauca asiste a prestigioso evento mundial de Ingeniería de Software el 27/05/2018. Se trata de la ingeniera Marta Cecilia Camacho, quien participa en Suecia como ponente en Workshop de Cooperación y Aspectos Humanos de la Ingeniería de Software dentro de la 40ª Conferencia Internacional de Ingeniería de Software.
news  popayan  phd  education  love  colombia  software 
may 2018
Python Popayán
Web page for the Popayán Python study group.
python  popayan  free  software  education 
april 2018
RethinkDB is the open-source, scalable database that makes building realtime apps dramatically easier. RethinkDB pushes JSON to your apps in realtime. This could have been a simpler solution for the invitations problem...
databases  storage  nosql  scalability  json  realtime  servers 
march 2018
Destroy All Software
Screencasts on various programming topics, like: testing, design, fluency with tools like Unix shells, computation - explained for programmers using code, but no mathematical notation.
blogs  programming  videos  cs  computation  reference 
march 2018
A binary alternative for doing RPC over HTTP/2. Advantages include: Define your service using Protocol Buffers, a powerful binary serialization toolset and language. Automatically generate idiomatic client and server stubs for your service in a variety of languages and platforms. Bi-directional streaming and fully integrated pluggable authentication with http/2 based transport.
http  rest  protocol  lib  services  distributed  google  streams  authentication 
march 2018
Algorithm Visualizer
Visualize code and animations for popular algorithms in diverse topics. A shame the code it's in JavaScript.
algorithms  animation  visualization  graphs  data-structures  tools 
february 2018
Fast JSON API serialization with Ruby on Rails
Fast JSONAPI (contributed by Netflix) is aimed at providing all the major functionality that Active Model Serializer (AMS) provides, along with an emphasis on speed and performance by meeting a benchmark requirement of being 25 times faster than AMS.
json  ruby  rubyonrails  optimization  performance  orm  api  lib  opensource 
february 2018
The Modular Monolith: Rails Architecture
Rather than extracting microservices, we decided to first focus on making our app modular. Our goal was to identify good architectural boundaries before we extracted code out into independent services. This would set us up to be able to migrate to microservices in the future, by having the code structured in a way to make a smooth transition.
ruby  rubyonrails  architecture  design  services  microservices  articles  howto 
january 2018
What I Wish I Had Known Before Scaling Uber to 1000 Services
What I wish I had known before scaling Uber to 1000 services. Yup, even Uber had some issues with microservices - not all technical, some are political. The "use the best tool for the job" philosophy leads to a culture fragmented around languages. Monitoring, performance, debugging, logging, fanout, load testing, RPCs, all sort of problems pop up when using microservices.
uber  microservices  scalability  monitoring  performance  go  java  python  tools  presentations  debug 
january 2018
SNES Classic Edition Manuals
All manuals for the SNES Classic Edition videogames.
nintendo  videogames  games  documentation  pdf  reference 
december 2017
NES Classic Edition Manuals
All manuals for the NES Classic Edition videogames.
nintendo  videogames  games  documentation  pdf  reference 
december 2017
Microservices on AWS
Whitepapers and resources for ElastiCache and Microservices, including:
1) ElastiCache design patterns, performance tuning tips,and important operational considerations to get the most out of an in-memory layer
2) Characteristics of microservices, challenges of building microservices, and how product teams can leverage AWS including ElastiCache to overcome these challenges
papers  aws  amazon  microservices  redis  architecture  memory  performance  design  patterns  tips 
december 2017
A Guide to Parsing: Algorithms and Terminology
In this article we make a more in-depth presentation of the concepts and algorithms used in parsing, so that you can get a better understanding of this fascinating world. We have tried to be practical in this article. Our goal is to help practicioners, not to explain the full theory. We just explain what you need to know to understand and build parser.
cs  concepts  parsers  algorithms  regexp  grammar  lexers 
december 2017
How to build a good API using RubyOnRails
This guide assumes you have a basic knowledge in programming and a good understanding of how an API works. It should be treated as a brief introduction to API development in Rails.
api  ruby  rubyonrails  rest  version-control  json  webapps 
december 2017
Building a graphql API in Rails
A tutorial on building a basic GraphQL API using Rails, with the recommended tools. With links to more advanced topics that you'll need for production, like connections, solving the n+1 problem, etc.
graphql  api  ruby  rubyonrails  rest  howto  tutorials  webapps  architecture 
december 2017
« earlier      
abandonware admin ai ajax algorithms amazon analysis apache api apple apps architecture articles assembler assessment asynchronous attacks aws bash blogs bogota books browsers bytecode c c# c++ cache cloud clustering code colombia comics commands comparison compilers complexity computation concepts concerts concurrency config constructs courses cpu cracking cryptography cs css data-structures databases debug design distributed documentation downloads dynamic eclipse education ejb emulation encryption erlang examples faq firefox formal forums framework free fun functional games gc geek google graphics graphs gui hacking hardware haskell heinsohn history howto html html5 http ide implementation install internet interpreters interview j2ee java javascript jee5 jpa jvm lambda languages lib linux lisp literature load logic mac machine-learning management mapreduce math media:document memory metal microsoft modeling monitoring music mysql networking news nintendo nosql oo opensource optimization oracle orm os osx papers parallel parsers patterns pdf people performance persistence photos php physics plugin portals presentations privacy profiling programming proof python quantum reference regexp research ruby rubyonrails scala scalability scheme sci-fi science scripting searching security servers setlist shell snippets soa social software space sql statistics storage syntax tappsi teaching testing text-processing threads tips tools tuning tutorials tv types unix version-control videogames videos visualization vm web web2.0 webapps webservices wiki windows work xml

Copy this bookmark: