Overview

This chapter talks about how we can store the data that we’re given, and how we can find it again when we’re asked for it.

There is a big difference between storage engines that are optimized for transactional workloads and those that are optimized for analytics.

We will talk about two families of storage engines: log-structured storage engines, and page-oriented storage engines such as B-trees.

Data Structures That Power Your Database

We can store data in a log and many databases internally use a log, which is an append-only data file. …

Overview

Most applications are built by layering one data model on top of another. Each layer hides the complexity of the layers below it by providing a clean data model. These abstractions allow different groups of people — for example, the engineers at the database vendor and the application developers using their database — to work together effectively.

Relational Model Versus Document Model

The best-known data model today is probably that of SQL, based on the relational model: data is organized into relations (called tables in SQL), where each relation is an unordered collection of tuples (rows in SQL).

The birth of NoSQL

The dominance of relational databases has lasted…

Overview

The main functionality of a standard system:

• Store data so that they, or another application, can find it again later (databases)
• Remember the result of an expensive operation, to speed up reads (caches)
• Allow users to search data by keyword or filter it in various ways (search indexes)
• Send a message to another process, to be handled asynchronously (stream processing)
• Periodically crunch a large amount of accumulated data (batch processing)

There are datastores that are also used as message queues (Redis), and there are message queues with database-like durability guarantees (Apache Kafka).

If you have an application-managed caching layer (using…

Overview

Server provisioning is the process of setting up a server to be used in a network based on the required resources. Provisioning can encompass all of the operations needed to create a new machine and bring it to a working state and includes defining the desired state of the system. Provisioning servers in a fast manner can dramatically improve the efficiency of an organization, especially for organizations that still heavily rely on manual processes.

This article highlights the detailed processes and tricks that can be used to provision servers (Virtual Machines in VMware Sphere). …

Highlights

1. Write InSpec Profiles once and use them everywhere
2. Test-Driven Development using InSpec Profiles
3. Test locally and build/release using CI/CD pipelines
4. Use chef_cookbook_generator to generate the cookbooks and policies
5. Use data bags even with kitchen test
6. Use Azure DevOps as repositories and CI/CD pipelines
7. Use kitchen-azurerm as the test driver
8. Use Artifactory to store cookbooks and binaries
9. Use Audit cookbook in Policies

Overview

Unlike other types of development, developing, testing, and releasing Chef stuff can be difficult and challenging. I am gonna walk you through how I minimize the work and maximize the efficiency of the development and deployment. …

When we practice solving coding problems, everyone may face problems that ask us to find the longest *** for the given input and logic, many know there are techniques to solve it, after solving quite a few them, I’ like to share what I have learned.

General Tips and Techniques

Generally, we can use the sliding window, two-pointer, and dynamic programming techniques to solve the problems.

The result can be calculated based on two cases.

1. If it is the length of consecutive elements (substring)
2. If it is the length of non-consecutive (may be consecutive as well) (subsequence)

1. If it is the length of…

1. Bitwise Operators

1.1 Basic Operations

1. AND ( & ): If both bits in the compared position of the bit patterns are 1, the bit in the resulting bit pattern is 1, otherwise 0. e.g. 5 (101) & 3 (011) = 1 (001).
2. OR ( | ): If both bits in the compared position of the bit patterns are 0, the bit in the resulting bit pattern is 0, otherwise 1. e.g. 5 (101) | 3 (011) = 7 (111).
3. XOR ( ^ ): If both bits in the compared position of the bit patterns are 0 or 1, the bit in the resulting bit pattern…

Table of Contents

1. Concurrency
2. Networking
3. Abstraction
4. Read-world performance and estimation
5. Map-reduce
6. Hadoop and Spark

1. Concurrency

1.1 Threads and processes

• A thread is a basic unit of CPU utilization. It is also referred to as a “lightweight process”.
• A process is an instance of a computer program that is being executed. It contains the program code and its current activity. Depending on the operating system, a process may be made up of multiple threads of execution that execute instructions concurrently.
• CPU is not required when a process is handling the IO request.

1.2 Concurrency and parallelism

• Concurrency means executing multiple tasks at the same time but not necessarily simultaneously. In a single-core environment…