Skip to content

Latest commit

 

History

History
344 lines (256 loc) · 36.2 KB

scaling.md

File metadata and controls

344 lines (256 loc) · 36.2 KB

drawing

How to over-engineer your next application, turn people into processes, and make billions along the way.

Back in the days when MySQL was the undisputed open source database champion, there was a sense that people who took their databases seriously would choose PostgreSQL instead. Everyone else chose MySQL; it was fast and it was easy.

In our last update, we discussed what sharding is, the problems it aims to solve and some of the hurdles we need to overcome to effectively implement it. Today we’ll look at cross-shard transactions, invalid transactions, and malicious behavior.

Take a look at the following chart:

Nothing beats the success story of a visionary leader who decides to take a bold course that sets the standard for how an industry operates. In the realm of technology, one of these stories is the infamous API mandate that Jeff Bezos, CEO of Amazon, sent out in 2002. For many technologists, the mandate wasn’t just the regular email, but it was the moment that the digital platform as we know it today was born. What is often overlooked, however, is that the interface-mandate (i.e., API-mandate) not only revolutionized internet technology development but it also changed how new business models came to be and how a wealth of new value was created.

Crypto exchange has been in great demand with the adaptability of cryptocurrencies surging and multiple new tokens/coins been put there attracting users by showcasing their great potential.

This is a collaboration between Baolong Mao's team at JD.com and my team at Alluxio. The original article was published on Alluxio's blog. This article describes how JD built an interactive OLAP platform combining two open-source technologies: Presto and Alluxio.

In my never-ending quest to find harmony between speed of doing things (development speed) and speed of the thing (performance), I came across a project in its incubation phases which had me thinking: “This could be the one language for me.”

In many cases, custom metrics may be useful to make more ad-hoc configuration of autoscaling in AWS. For example, a custom metric can allow an autoscaling group to react more rapidly on spikes or to take into account the health state of hosts for concurrency based metrics. Let's see how to set up a custom metric and autoscaling policy for an autoscaling group with an application load balancer and target group using AWS Console.

Want to add better visibility and flexibility to your microservies architecture? Read on to find out how adding a Conductor can help!

Laravel Octane is a great way to scale your Laravel application. It's easy to use and can improve performance without needing to make changes to your codebase.

The evolution, failures and design decisions behind one of the world’s largest real-time, high-frequency and low-latency streaming systems.

At its heart, scaling a team is about matching skills to problems. In this piece, we share a guide about scaling backend API software development teams

There is one of those "universal truths" in the world of startups that, like almost all of them, is neither as universal nor as TRUE as we usually think, and that investors often repeat: 9 out of 10 companies fail.

Deep dive into how Zerops, a dev-first cloud app platform, auto scales your apps vertically and horizontally. Runtimes, databases, object storage, and more.

Here's a simple example of how you can automate scaling image transformations. Tech used: Node.js, Sharp, Express. App platform: Zerops

Imagine you are in the process of migration of a legacy system from the data center to the cloud to make the system scalable. If containerization is not an option you will try cloud-native autoscaling. One of the most commonly used metric to perform autoscaling is CPU utilisation. If your application does not scale well based on this metric this is pretty likely because of more serious issues. Let's see why.

When it’s time to scale up/down, time is money. See how easy it is to scale your applications and services using Render.

Last year, Uber announced that it will start showing Lime scooters and Jump bikes in Uber app. On surface, it looks like this move will cannibalise Uber’s cab business. Per trip revenue made by Uber is much higher in case of a cab than in case of an e-scooter or a bike.

Find out how to set up processes, effective development practices & communication strategies to make design decisions and handle tech debt as the team scales.

I’ve been in and out of the blockchain ecosystem for the last three years. I’ve made my first purchase the same month Binance launched. In the meantime, I’ve gained a CoinMarketCap compulsive checking behavior after holding a bag through the last crypto winter.

How might Salesforce developers build apps that execute long-running commands which can scale with user growth while not impacting performance?

Moving towards digital communication at scale has always been a tedious task for all types of businesses, no matter the industry you're serving or the size of your business. With so many technological advances companies are now looking for new ways to enhance and improve their customer experience and at the same time to save in operational costs.

Often in the startup world, you need to spend money to make money. My business partner, Roz Lemieux, and I raised our second round of funding, Series A, on the strength of our belief that we could expand the appeal of our product, Attentive.ly, to the private sector. 

Scaling blockchain by adding, not removing.

Factors to consider in scaling your enterprise solution: the do's , the don'ts and the maybe.

A look at how small businesses can scale quickly and relatively cheaply by using digital methods for marketing and other such technologies to their advantage.

Fourth annual report reveals continued market maturation; $34.5 billion invested in southeastern startups since 2015

Growing a company is tough, and it can become even more difficult if entrepreneurs don’t ask the right questions.

You don't need a massive budget to create world-class design. Learn from these case studies how design can help startups scale.

Brian Scudamore | Scaling Secrets: It’s All About People

In cloud computing, cloud scalability refers to the ability of increasing or decreasing the IT resources to meet changing needs. Scalability is an element of the cloud and is the main driver of its popularity in the business sector.

An analysis of cross- chain asset bridge mechanisms. Ethereum scaling solutions Polygon vs optimistic and zk rollups compared

Guys! During the YC Startup School I gradually started to implement different ideas to track primary metrics and I have involved my team to set OKR (Objectives and Key Results) and track their important metrics too. Eventually it has been structured on a dashboard using just google spreadsheets and formulas and charts.

Radix is the first programmable DeFi Engine for the smart contract platform that helps Dapp scale their products and services.

Scaling web applications is a skill. It’s like walking a tight rope, in the midst of a storm. There are multiple factors acting on you,

Recently, AdPushup engineers had a proud moment when their in-house system auto-scaled 10x with 1.2B+ ad feedback requests and 425M unique impressions per day.

It doesn’t matter if you are running background tasks, preprocessing jobs or ML pipelines. Writing tasks is the easy part. The hard part is the orchestration— Managing dependencies among tasks, scheduling workflows and monitor their execution is tedious.

Working for a company that builds machine learning software for robotics, I hear these questions every day. Machine learning has become a shiny object that everyone wants to pursue. Over 80% of the companies are looking into at least one AI project.

A CDN designed for APIs with pre-heating and real-time purges, that sits in front of all our API servers, always serving a hit and never a miss.

The decision to welcome the help of outside investors is never an easy one, so it seems unfair that after most businesses make the decision of turning to venture capital they then have to embark on an arduous path towards gaining a windfall. 

What your business can learn from Silicon Valley’s approach to grow customers

Ever find yourself maintaining a prototype in production? Or struggling to scale siloed products? In this article, we explore a framework for scaling software.

How to make your WordPress site safe and fast by using Amazon CloudFront to keep your site private and harder for malicious actors to access and tamper with.

The circuit breaker is a design pattern, used extensively in distributed systems to prevent cascading failures. In this post, we’ll go through the problem of cascading failures and go over how the circuit breaker pattern is used.

We often work on building applications where a single application might scale to multiple instances with couple of changes. A good example for this case will be, when you are building an application for multiple countries. TLDR; Different countries might have different requirements i.e different sections or different form fields, but the base application i.e common part should remain same.

This post is part of the Hacker Noon Shareholder Series, where we interview some of the super-investors who made the site you're on right now possible.

The art of building a large catalog of connectors is thinking in onion layers.

At Debricked we have been using Symfony for our web backend for a while now. It has served us very well and when they announced the Messenger component back in Symfony 4.1, we were eager to try it out. Since then we have used the component for asynchronous queuing emails.

Certain industries greatly benefit from high-performing, low-latency, geo-distributed technologies.

By Nikolas Vogt Founder, Growth Academy | Ex-Growth Lead, Google Assistant | Guest speaker, Santa Clara University

In January 2019 I launched Truepush, a free forever platform for Push Notifications, on ProductHunt. The very next morning after trending as the “Product of the day”, the skyrocketing growth was set to take place in the next coming months. A lot of growth strategies were implemented and continuous feature updates were done covering the journey of one whole year.

Every time facing a task of configuring how the system should scale out and scale in, we decide what metrics and policies will be more effective. I have already described CPU based policies and what challenges they may have. But what if this is not an option, i.e. the service is represented by legacy application, written years ago, not well documented, etc. and we still want to run it in Cloud and autoscale?

Silicon Valley has always been a dream destination for developers and tech talent. The so called “The Bay Area,” is the world’s largest and most famous startup ecosystem.

Last year, after a bit of wrangling and lots of editing by the fantastic Jenn Webb, O’Reilly published a discussion Mark Burgess and I had on one of his trips through the Valley as a podcast.

Hundreds of startup founders around the globe choose to incorporate their companies in the US to get access to the flourishing startup ecosystem.

One of the most promising foundational constructions to build layer 2 solutions are optimistic rollups. Many projects are built upon them

Your website failed under load from online traffic. What went wrong? There’s usually not a single flaw, but odds are 1 of these 6 technical flaws is at fault.

Top picks of growth hacking visuals curated by Growth Academy — The growth strategy course by leaders from Silicon Valley

Spawning an AWS EKS cluster has never been easier and options are many: CloudFormation, Terraform or CDK. For the lazy, you can even use the great CLI utility eksctl from Weavework.

What’s so great about RChain (in plain English)

Most agile teams do sprint retrospectives at least once a month, to iterate and improve on their software development process and workflow. However, a lot of those same teams rely only on their feelings to “know” if they have actually improved. But you need an unbiased reference system if you want to compare how two sprints went. 

With the introduction of the Bitcoin network software, a new era of technology was born. It provided, for the first time, a trustless, peer-to-peer payment network with a very transparent, and rule-based money supply. It revealed a distributed ledger technology that will profoundly change the way society transfers value, and ushered in a new era of trusted decentralization.

As we move into the cloud computing era, we've seen the floodgates open and waves of new development frameworks taking advantage of every opportunity that cloud computing technologies offer.

Like Web3, the concept of "metaverse" does not yet have a clear definition.

You're a scooter-sharing start-up who just raised $100 million from Seq-reessen-losa Ventures. Cool. It's time to drop those 50,000 scooters you just unloaded from container ships fresh from Shenzhen. You have 100 cities on your todo list. What's your plan to keep track of them once you've unleashed them on millions of unsuspecting citizens?

Ruby on Rails is a tremendous framework when you want development speed for your project or startup. It’s useful right out of the box and comes with a plethora of behind-the-scenes magic to make your life easier. However, it’s not considered the fastest framework out there in terms of performance. You will find examples of individuals and companies drifting away from Rails in favour of something else. Despite this, there are many companies out there who have succeeded in scaling Rails and found success — just take a look at Airbnb, Github, Gitlab & Shopify. 

Imagine the situation, you have a legacy system partially migrated to multiple services and used by the relatively small amount of concurrent users. And one day stakeholders want to create an open API and expose the system to external clients with the load significantly higher than the existing system can handle, and it should be scalable to handle in the future even more.

Emil first sketched the idea for the company on the back of a napkin during his flight to Mumbai in 2000. Today, the company is the world’s leading graph platform, powering mission-critical enterprise applications, including artificial intelligence, fraud detection, real-time recommendations, and master data.

From a business perspective, 2020 has made one thing clear: the world is going digital at an accelerated rate. If you think about it, this is the first time in history when most business is happening virtually, offering a unique position to companies worldwide to create a footprint and legacy like never before.

It's early on in a products lifecycle that it begins to require a global SRE presence. Once you have gained sufficient customer traction, and if your product warrants it, you need to provide them with around the clock availability support to complete the customer support jigsaw.

Best Practices of Serverless at Scale 

If you were ever searching for a job during the past two decades, there’s a good chance you’ve used an online job board; a website that lists jobs supplied by employers. Such a “board” could’ve been found on shop windows, in newspapers, and now on the Internet.

In this article we will cover the core concepts of Kafka and also will touch upon a few of the advanced topics.

Don't get trapped in the Death Star! Choreographed microservices have all the benefits of a loosely coupled architecture, and only some of the pitfalls.

Lean methodology is simple: identify value, and deliver it with as few steps as possible.

In the race for $ETH scalability, the Layer Wars are in full swing. Will the future of scalable decentralization rely on #Ethereum 2.0, other L1s, or L2s?

From load tests to feature toggling to performance profilers, discover 13 expert tips to build performance into your web application.

  1. Do I Understand the VC Asset Class and Perspective?
  2. Am I Building a Predictable Customer Acquisition Model?
  3. Is My Startup a Billion Dollar Company?

How dotNetify implemented horizontal scaling of SignalR apps with proxy instead of a backplane.

How to build scalable products by coding for future feature expansion & code deprecation while maintaining a simplified user interface for complex functionality

I’m pretty jaded when it comes to the financial numbers that get thrown around in start-up land—I don’t care what TechCrunch says, I don’t care what your market cap is—I think it’s clear that the world of technology start-ups is pretty screwed up when it comes to financial valuations. Only in this world is whether or not your company is profitable so often seen as being of secondary importance. I digress.

The slow death of the old ways