Dear developers and system administrators, we need your expertise in comprehending Docker's functionality and how to set it up. Take my situation for example: I have a VDS with Debian and I need to install MySQL(MariaDB), Python/Django, Nginx, Memcached, and Sphinx Search. What I hope to accomplish with Docker is to segregate the resources of some resource-intensive components, experiment with various settings and versions of components, and easily shift containers to a new host if necessary.

However, despite all the manuals and explanations I've read over the past few days, I still haven't found an answer to my problems. All I've found so far are hints and vague lines on how to do it but nothing comprehensive - no step-by-step instructions on how to set it up. Thus, I have some questions that have been left unresolved:

1) Containers in Docker are stateless, but I still need to store and create data. How can I create, store, and record changes that I make?
2) Is there an easy way to turn stateless containers into stateful ones so that I can start them automatically with changes? For instance, if the server crashes and starts again from where it left off?
3) What is the best way to break down my server into more manageable containers?
4) How do I choose the necessary application containers on Dockerhub?
5) Where is the best place to store data (like Django projects) so that I don't lose anything, but I can conveniently migrate to another host if necessary?
6) How can I safeguard sensitive data from being leaked to Dockerhub along with my container image?

Additionally, it would be great to hear any advice or suggestions that you might have on this topic.

You may have gone through multiple manuals, but the essential dоcument - the official dоcumentation - might have been overlooked. What does it contain? It explains how and why things should be done. Although the questions have already been answered, I will reiterate them since I spent a considerable amount of time reading through the answers.

The official dоcumentation can be found using the following links:
1) https://docs.docker.com/engine/userguide/dockervolumes/
2) https://docs.docker.com/engine/articles/host_integration/

The answer to the question of how many components can be in one container is quite straightforward - it's up to you. Although it might sound cliché, you know what's best for your project. It is recommended, however, that one component per container be used. This approach makes sense so that only the container containing the updated component needs to be updated without worrying about potentially breaking other components.

If you're still unsure about how to proceed, write your first Dockerfile to get a better understanding. Until then, trust only official images.

Regarding the question about git, this is likely a result of a misunderstanding from question 1). The answer has already been provided. If you're having trouble grasping the concept, it's best to either avoid using dockerhub altogether or put in the effort to understand it fully. Alternatively, consider paying for private repositories to avoid the hassle of uncertainty.

Regarding the management level of the infrastructure, there is a lot of buzz around Google Kubernetes these days. However, personally, I find the less popular Rancher to be a very impressive platform. In fact, I believe it will be a great fit for you since it offers the following capabilities:

a. It allows you to connect to a single web interface for controlling machines across various cloud providers.

b. It enables you to manage most parameters directly for both containers and larger bundles.

c. It offers the ability to manage volumes for persistent data storage, and the issue of transferring data between hosts can be easily resolved by lifting the GlusterFS cluster in just three clicks. It also has its own development for the sink -- Convoy.

d. It provides the ability to monitor the functioning of services and hosts by automatically launching containers on other hosts.

e. It allows you to create your own private network between hosts, even if they are located in different data centers from different providers.

f. It facilitates load balancing between several containers on different cars.

Configuring Rancher to achieve all these features is not rocket science. However, as a starting solution that hides many setup challenges, it should work well for you.

Databases can be stored in normal containers such as lxc/openvz or in a separate VPS, mainly due to the fact that backups and replication can function in normal mode with normal access. Another advantage is that the ip address is static, which is not available in Docker and unlikely to be added.

Docker containers are virtual machines designed to run a single process. You specify in the dockerfile what should be performed, and what specifically needs to run in the foreground. If the application runs in the background, the container will stop, as the service apache2 start option does not work. Therefore, it's important to read about the Docker pid 1 problem and why it all starts in the binding from the same bash.

Moreover, Docker heavily relies on its hub and git for "git pull & start app with one button" on some kind of internal IP address from the gray network.

I'll do my best to provide answers and advice based on your questions.

1) To create, store, and record changes in Docker containers, you have a few options. You can use persistent storage volumes or bind mounts to store data outside the container and make it available to multiple containers. Docker provides various ways to manage data, such as using named volumes or specifying host paths. This allows you to retain data even when containers are stopped or removed.

2) While containers in Docker are typically designed to be stateless, you can make them stateful by leveraging persistent storage. By using persistent volumes or bind mounts, you can ensure that changes made within containers are preserved even if they are restarted or moved to another host. This way, you can resume where you left off after a server crash.

3) Breaking down your server into containers depends on your specific application architecture and requirements. Generally, it is recommended to divide services into separate containers, such as running MySQL/MariaDB in one container, Python/Django in another, and Nginx in a separate container. This approach allows for better isolation, scalability, and easier management of individual components.

4) Dockerhub is a popular repository for finding and sharing Docker images. To choose the necessary application containers, you can search Dockerhub for official images provided by the software developers themselves or well-maintained community images. Look for images with detailed dоcumentation, recent updates, and a significant number of downloads or stars.

5) To store data like Django projects, it is advisable to use persistent storage. Docker offers various options, such as volume mounts or bind mounts, which allow you to keep your data separate from the container and easily migrate it to another host if needed. It's important to ensure that your data is backed up regularly to avoid any potential loss.

6) Dockerhub allows you to store images publicly or privately. To safeguard sensitive data, you should avoid including sensitive information in your container images. Instead, use environment variables or mount external configuration files during runtime to provide the necessary secrets securely.


additional tips and suggestions related to Docker that might be helpful:

1) When choosing containers from Dockerhub, it's important to consider factors such as the popularity of the image, the community support, the number of contributors, the recentness of updates, and the security history of the image. These factors can help ensure that you are using a reliable and well-maintained container image.

2) To manage multiple containers efficiently, you can use Docker Compose. Docker Compose allows you to define and manage a group of related containers as a single application. This simplifies the process of starting, stopping, and managing multiple containers that work together.

3) Consider using an orchestration tool like Kubernetes or Docker Swarm for managing container deployments in a production environment. These tools provide advanced features for scaling, load balancing, self-healing, and managing containers across multiple hosts.

4) As your deployment grows, monitoring and logging become essential. Look into tools like Prometheus, Grafana, and ELK Stack (Elasticsearch, Logstash, Kibana) to collect and analyze metrics and logs from your containers. This will help with troubleshooting, performance optimization, and maintaining the health of your system.

5) Security is a crucial aspect of containerization. Ensure that you keep your host system up to date with security patches, follow best practices for network security, and properly configure access controls for containers. Limit unnecessary privileges and regularly scan the integrity of your container images and the host system.

6) Explore options for container backup and disaster recovery. Having regular backups of your data and configurations will help you quickly recover from any potential failures or incidents. Docker provides tools like docker commit, docker export, and docker save to create snapshots of containers or export them for easy backup and restore.

Remember, Docker is a powerful tool, but it does require some learning and experimentation to fully understand and utilize its capabilities. Don't hesitate to explore the vast Docker community, attend meetups or conferences, and engage in discussions to share knowledge and learn from others' experiences.

Containers are stateless, and if you're not using volumes, you're setting yourself up for data loss. You want stateful behavior? Docker Compose is essential for managing dependencies and ensuring your containers restart with their data intact.

When selecting application containers, don't just grab any image from Docker Hub, scrutinize them for quality and support. For your Django projects, store data in volumes or external storage to facilitate migrations. And let's not forget security - never expose sensitive data in your images. Use environment variables or Docker secrets to keep your credentials safe. If you're not doing this, you're asking for a security breach.