There exists a group of individuals who place high value on reliability, stability, security, and freedom of information. These individuals are likely those who establish Plex and Jellyfin media web servers, launch Bitcoin nodes, Tor bridges, Mastodon and Matrix instances, YunoHost applications, Tailscale VPN nodes, etc. This seems to be a natural process.
(https://www.sevenlayers.com/images/blogimages/2020/MyWeb/image0002.png)
Decentralization, peer-to-peer communication, autonomy, and self-hosting are the fundamental principles at play here. The goal is maximum independence from environmental conditions, states, banks, and other external factors. When you have a solid foundation and sovereign autonomy with financial independence, the problems of third-party services become secondary. Although failures in their work can be inconvenient, they are not critical if backup options are available.
People often ask whether raising their own server is difficult. In reality, the complexity of this task is often overstated. Let's take a look at a simple example of a server.
The simplest server is typically just one line in the console:
while true;
do echo -e "HTTP/1.1 200 OK\n\n$(iostat)" \
| nc -l -k -p 8080 -q 1;
done
The most basic home NAS can be an old smartphone or a single-board computer costing only five dollars, connected to a hard drive, microSD, SSD, etc. So it's not as complicated as it may seem.
A web server is essentially just a script. One interesting example of a web server made from scripts is Sherver. It is a pretty advanced and multifunctional server written entirely in Bash, an improved version of bashttpd in some ways.
The server is made up of several scripts, the first of which is ./sherver.sh. To bring up Sherver, clone the repository and run ./sherver.sh. You can specify the working port at startup: ./sherver.sh 8080 (this is the default).
That's it. After that, open your browser and go to http://localhost:8080/ – everything works. The site loads from html, images and other resources placed in the /files folder.
In essence, the entire concept of a server is a straightforward and even primitive thing. If someone tells you that it is difficult to raise a server, show them a script like this. The following tools must be present in the system for Sherver to be operational:
- envsubst, if you want to use templates
- socat for server operation (netcat is also possible, but it does not cope very well with parallel HTTP requests)
Sherver operates over HTTP 1.0 protocol and is best suited for providing multiple pages over the internal network. It will not withstand heavy loads and must be secured with a firewall to prevent intrusions from outside.
Strictly speaking, Sherver should not be exposed to the internet at all, as it would likely end in disaster. Essentially, we would be transferring the system console to the client's browser.
Sherver can be used either as an emergency "plug" in specific situations (see below about the creation history), or as a web server in an internal network closed from the outside. For example, in a home or corporate network.
How could a novice suggest this as a straightforward web server option? More popular options like nginx or Apache would be easier to set up and function well. A few years ago, "advanced" IT managers claimed that everything should be stored in the cloud, touting its cost-effectiveness with impressive figures. However, we are now seeing a trend of companies frantically downloading gigabytes of archives onto local servers.
A web server is essentially a computer program that can be installed on any operating system to process HTTP requests and issue HTTP responses, which occur over the HTTP protocol and allow communication between the web server and browser. When an address is entered into the browser's address bar, a request to the web server begins to be executed. The web server must then process these requests and returns a response containing all the necessary information for the browser to display the web page. It is also responsible for creating a log of errors and file accesses, as well as authentication and authorization of users. The web server contains settings about how to process files and folders on the server. Multiple companies develop web servers with their own algorithms that work differently.
Server independence refers to the concept that the functioning and performance of a server should not be dependent on any specific hardware or operating system. It emphasizes the ability to abstract the server's resources and services from the underlying infrastructure, enabling easy migration, scalability, and management.
There are several basic principles that guide server independence:
1. Virtualization: Server virtualization is a key technology that enables server independence. It allows multiple virtual machines (VMs) to run on a single physical server, abstracting the hardware resources and providing isolation between different VMs. This abstraction enables easy migration of VMs across different physical servers without affecting their functionality.
2. Abstraction Layers: Server independence relies on the use of abstraction layers to separate the applications and services running on the server from the underlying hardware and operating system. This abstraction enables applications to remain unaffected by changes in the hardware or software environment, ensuring consistent performance and availability.
3. Standardization: Standardizing server components, interfaces, and protocols is crucial for achieving server independence. By using open standards, such as TCP/IP for networking and HTTP for web services, servers can communicate and interact with each other regardless of their underlying hardware or operating system.
4. Scalability: Server independence facilitates horizontal scalability, which means adding more servers to handle increasing workloads. With server independence, new servers can be added to the infrastructure seamlessly, without impacting the applications or services running on existing servers.
5. Automation and Orchestration: To achieve server independence, automation and orchestration tools play a crucial role. These tools automate the deployment, configuration, provisioning, and management of servers across various hardware and software environments. They enable the efficient management of server resources, ensuring high availability and quick response to changing demands.
6. Decoupling: Server independence promotes the decoupling of applications from specific servers. This means that applications should be designed to function independently of the server they are running on, enabling easy migration and deployment across different environments.
7. Multi-cloud and Hybrid Environments: Server independence extends beyond a single cloud provider or infrastructure. It encompasses the ability to deploy and manage servers in multi-cloud or hybrid environments, where applications can run seamlessly across different providers or a combination of on-premises and cloud resources.
8. Service-Oriented Architecture (SOA): Adopting a service-oriented architecture approach promotes server independence. By breaking down applications into modular, interoperable services, it becomes easier to migrate and scale individual services independently, without disrupting the entire application.
9. Continuous Integration and Deployment: Server independence aligns well with DevOps practices, particularly continuous integration and continuous deployment (CI/CD). CI/CD pipelines enable rapid provisioning and deployment of applications while maintaining server independence by automating build, testing, and deployment processes.
10. Security and Compliance: Server independence should not compromise security and compliance requirements. Secure configurations, access controls, and adherence to regulatory standards should be maintained regardless of the server environment. Server-independent solutions should support secure communication, data protection, and monitoring capabilities.
11. Monitoring and Management: A robust monitoring and management system is essential for server independence. It allows administrators to track the performance, health, and availability of servers across different environments, ensuring efficient utilization of resources and proactive issue resolution.
12. Disaster Recovery and High Availability: Server independence should include mechanisms for disaster recovery and high availability. Redundancy and failover capabilities should be built into the infrastructure to ensure continuity of service in the event of server failures or disasters.
13. Containerization: Server independence can be further enhanced by adopting containerization technologies such as Docker or Kubernetes. Containers encapsulate applications and their dependencies, making them portable and easily deployable across different server environments.
14. Immutable Infrastructure: Adopting an immutable infrastructure approach supports server independence. In this model, servers are treated as disposable entities that are replaced instead of being patched or modified. This ensures consistent deployment and eliminates dependency on specific server configurations.
15. Infrastructure as Code (IaC): Server independence is closely linked to the concept of Infrastructure as Code, where infrastructure resources are defined and managed using code. Through tools like Terraform or CloudFormation, IaC enables the provisioning and management of servers in a consistent and repeatable manner, irrespective of the underlying infrastructure.
16. Microservices Architecture: Server independence aligns well with a microservices architecture. Breaking down applications into smaller, loosely coupled services promotes flexibility, scalability, and portability. Each microservice can run independently on its own server, enabling easy deployment and scaling.
17. API-Driven Development: Server independence can be facilitated by adopting API-driven development practices. By exposing functionalities through well-defined APIs, applications can interact with servers and services in a standardized way, regardless of the underlying technology stack.
18. Vendor Neutrality: Maintaining server independence requires avoiding vendor lock-in. Organizations should strive to use open-source solutions and technologies that are not tied to specific vendors. This ensures freedom to switch between different providers or technologies without disruption.
19. Version Control and Configuration Management: Keeping server configurations and infrastructure code under version control is essential for maintaining server independence. This allows administrators to track and manage changes, roll back to previous configurations if needed, and ensure consistency across different server environments.
20. Continuous Improvement: Server independence is an ongoing process of continuous improvement. It requires regular evaluation of the infrastructure, identification of bottlenecks or dependencies, and implementing optimizations to enhance flexibility, efficiency, and resilience.