If you like DNray Forum, you can support it by - BTC: bc1qppjcl3c2cyjazy6lepmrv3fh6ke9mxs7zpfky0 , TRC20 and more...

 

Webserver configurations depending on expected loads

Started by sumoncps, Dec 14, 2022, 12:56 AM

Previous topic - Next topic

sumoncpsTopic starter

My aim is to establish a self-made service with simple functionality, which involves me taking charge of writing code. I'm not particularly familiar with web tools but after some research, I've settled for Python + Django.

One question that arises is what hosting services should I choose for my Python application, and what configurations should I consider, such as memory capacity, number of websites, and databases.

Another concern is the legal regulation of internet services. Can I confidently develop an app, put it into production without risking any legal troubles, such as copyright infringement or explicit content?

In regards to further guidance on the development process, hosting selection, server management and problem-solving, links to stories or articles sharing personal experiences would be greatly appreciated.
  •  


dragon

Starting from scratch is unnecessary. Hosting is suitable for nearly anyone with a python and database. However, the most optimal approach involves utilizing hosting with VPS to personally include everything required, ranging from the operating system and base to python and Django.

Personally, I employ digitalocean for development purposes. It's inexpensive and practical, with every resource necessary readily available. Additionally, the incorporation of an internal network between virtual machines provides an added layer of convenience.

It's recommended that one writes their application first before selecting their hosting, dedicated server or rack. For more information on how to use the Django One-Click Install Image for Ubuntu 14.04, visit: https://www.digitalocean.com/community/tutorials/how-to-use-the-django-one-click-install-image-for-ubuntu-14-04
  •  

lazy1boy

Estimating the expected load is most easily done based on the number of users connected simultaneously, but you should also consider future scaling requirements and plan for additional performance.

To achieve this, we recommend using the latest builds from leading server equipment manufacturer, Hewlett Packard Enterprise. Their products help to track the evolution of equipment as requirements increase.

For 10 or fewer connections, a basic system with at least a dual-core chip operating at a frequency of 2.2 GHz, 4 GB DDR4 memory, and three medium SAS/SATA drives is sufficient. The MicroServer Gen10 Plus - (P18584-421) is an excellent choice, with an Intel Xeon E-2224 processor with four cores clocked at 3.4GHz, and supporting four LFF disks.

For 10 to 20 users, a system with a quad-core processor operating at a frequency of 2.3 GHz, several DDR4-type RAM strips with a volume of 8 GB, and at least three SAS/SATA disks with a spindle speed of 7200 RPM would be necessary. The ProLiant ML30 Gen10 - (P16929-421) tower is a great option, including a 3.6GHz quad-core Intel Xeon E-2234, a 16GB RAM bar (expandable to four), and a 4LFF basket.

For 20 to 50 clients, an eight-core processor with 16 GB of RAM, and at least six SAS/SATA interface drives with a speed of 7200 RPM is required. The ML110 Gen10 - (P21440-421) tower is perfect for this load, with an Intel® Xeon Silver 4208 CPU with 8 cores clocked at 3.6GHz, a 16Gb memory bar (expandable up to 16), and a basket for 8SFF disks.

For 50 to 100 connections, a system with two octa-core chips operating at a clock frequency of 2.8 GHz or higher, DDR4 memory of 32 GB or more, and ten SAS disks with increased rotation speed is necessary. For instance, the two-socket rack DL360 Gen10 - (P19774-B21) with Intel® Xeon Silver 4208 CPU and an 8/10+1 SFF basket would be ideal.

For 100 to 200 users, a system with two sockets occupied by 16-core processors with a frequency of 2.8 GHz, 64 GB DDR4 RAM and 16 SAS drives with a speed of 10,000 RPM would suffice. The powerful two-socket DL380 Gen10 - (P20245-B21) with Intel® Xeon® processor Gold 6242 for 16 cores and a basket for 8/24+6 SFF 2.5" HP is a great option.

For 200 to 500 clients, a configuration with four 16-core chips with a frequency of 3 GHz, a RAM capacity of over 128 GB, and 20 or more SAS drives with a spindle speed of 10,000 RPM is required. The magnificent four-socket DL580 Gen10 - (P21273-B21) with Intel® Xeon® Gold 5220 with 18 cores and a huge basket up to 48 drives, coupled with 48 slots for 64 GB lines, is more than capable of handling such demanding projects.

In order to ensure uninterrupted access to valuable information, it is recommended to use a cluster of such web servers, while taking into account the specifics of the selected DBMS.
  •  

AuroINS111

When choosing a web hosting service for your Python + Django application, you want to ensure the service provider supports Python and has solid infrastructure. Some popular choices include:

DigitalOcean: Offers simple, flexible hosting solutions. DigitalOcean has a good track record with developer communities, cost-effective, and provides features like SSD hard drives and team accounts.

Heroku: It's very simple to deploy a Django app on Heroku. It's free to a certain degree and is appropriate for small projects.

AWS (Amazon Web Services) Elastic Beanstalk: Suitable for large-scale apps. AWS provides a wide variety of services and limitless scalability but it can be complex to use if you're not familiar with its architecture.

Google Cloud: Works well for applications that expect large amounts of traffic. It scales easily, but like AWS, its complexity can be difficult for beginners.

PythonAnywhere: It's very beginner-friendly and specifically supports Python web applications but might not be as robust for larger-scale applications.

Configuration Factors:

Every project and their needs are different, thus configuration needs can highly vary. However, there are few key configurations considerations for most web services:

Memory Capacity: It depends on the size and complexity of your application, the number of users you expect to have and the type of workload. Start small and scale vertically (increase server capacity) or horizontally (add more servers) as necessary.

Number of Websites: This depends on how many websites you plan to host. Some apps may require multiple websites (for example, a public-facing website and an internal admin dashboard).

Databases: Django comes with a built-in database abstraction layer that can work with various databases like SQLite, PostgreSQL, MySQL, etc.

Legal Regulation:

As for legal regulations, you'll need to consider the following areas:

User data protection regulations, such as the European Union's General Data Protection Regulation (GDPR) or California's Consumer Privacy Act (CCPA), depending on your user's location.
Copyright laws, avoid using or hosting copyrighted material without the proper licensing or approval.
Make sure to utilize secure protocols to protect your users' data from potential breaches.
Various countries have their specific laws; you might need to consider localization and regional laws if your application is international.
Development Process, Hosting Selection, Server Management:

For detailed guides on these topics, you can refer to the official Django dоcumentation and hosting service dоcumentation. They usually have detailed, step-by-step guide on getting your Django app up and running on their platforms. Here are few you might find useful:

Django Official Deployment Checklist
DigitalOcean Django Deployment Guide
Deployment on Heroku
Personal Experience:

Here are a few personal experiences and stories from developers which you may find informative and insightful:

Deploying a Django Application to Elastic Beanstalk
A 10 minute guide to deploying Django on DigitalOcean
Deploying Django on AWS ECS


In terms of additional web hosting options, you could consider some more like:

Linode: Similar to AWS, it offers a large range of selectable instances (including high-memory instances), data storage, and networking options. However, it may require more work from you to manage the system than a platform like Heroku.

Azure: Microsoft's cloud solution supports Python and Django, and has a range of services comparable to AWS. Plus with comprehensive dоcumentation and many user guide resources, it could be a viable choice.

It's crucial to assess your own project needs and requirements in order to make an informed choice about hosting. If you plan to have a large user base from day one, or if you prioritize minimizing costs, these factors will inevitably shape your decision.

Regarding further management and server considerations:

Backup: Regularly backup your database and codebase to ensure safety against any technical mishaps or data breaches.

Monitoring: Use tools like New Relic, DataDog, or Kibana to monitor your application's performance. Monitoring can alert you of any issues before they become major problems.

Security: Ensure your server and application are secured. Regularly update and patch your systems, implement measures like firewalls, and manage permissions and authentication rigorously.

Optimization: Over time, you may need to optimize your server configuration or even your application code depending on your app's usage patterns and performance characteristics. Profiling tools can help you spot performance bottlenecks and address them.

For Legal and Regulatory concerns:

Content: Be aware of the kind of content that gets posted in your app, especially if it's user-generated. Make sure it doesn't infringe on anyone's rights and doesn't contain any form of hate speech, pоrnographic material, or anything generally considered illegal.

Accessibility: In certain areas, including the US, web services are required by law to be accessible by people with disabilities. You might need to check the regulations around that.

Licenses: If you use any third-party libraries or software, ensure you adhere to their licensing terms.

Tales from other developers:

Deploying a Django Application to Google Cloud

This Reddit post discusses several developers' experiences with Django deployment and hosting.

And Finally this Quora thread has discussions on choices of hosting providers for Django apps.


More Hosting Options:

A2 Hosting: Known for its speed and reliability, A2 Hosting offers Python support and a variety of packages suitable for different needs.

Vultr: This is a cloud service that provides an infrastructure resembling a public cloud. They offer competitive pricing and a solid performance.

Further Configuration Considerations:

Python Environment: Make sure the hosting provider supports the Python version you're using and allows for environment isolation (e.g., using virtual environments).

Static and Media Files: Django does not serve static and media files in production. You will need to set up this separately. Consider using a Content Delivery Network (CDN) for improved performance.

HTTPS: Encrypting data in transit is extremely important for web applications. Your hosting provider should allow HTTPS connections, better if they provide free SSL certificates through Let's Encrypt.

Scaling: As your userbase grows, the website will sustain a higher number of requests and will need to uphold optimum productivity. Check how easy it is to scale resources on your chosen platform.

More Legal and Regulation Aspects:

Payment Systems: If your app is going to handle payments and sensitive user data, you have to comply with a set of Payment Card Industry Data Security Standards (PCI DSS).

Children's Online Privacy: If your service is intended for children below the age of 13 (in the US) you need to comply with the Children's Online Privacy Protection Act (COPPA).

Health Information: If your app handles health-related information of US citizens, compliance with Health Insurance Portability and Accountability Act (HIPAA) is mandatory.

Learning Materials & Developer Stories:

More on Deploying Django on Azure

More complex use-cases: Django and Kubernetes and here is another great article by a developer Django & Kubernetes: The Fully Illustrated Guide

Django deployment using GitLab, Docker, and Traefik: Django Deployment Done Right

Remember, ideally, there's no one perfect choice for everyone. Consider your project requirement, technical comfort, the price point, and what trade-offs you're willing to take (initial setup simplicity vs more control, cost vs scalability, etc.), and you will find the option that best satisfies your needs.

Lastly, always keep security as a top priority, especially when handling user data. The best practices you should follow include using HTTPS, keeping your software up-to-date, following the principle of least privilege, and hashing and salting user passwords.
  •  

David5

Services like Heroku, DigitalOcean, or AWS are popular among developers. Heroku offers simplicity and scalability, making it ideal for beginners. DigitalOcean provides more control and is cost-effective for small projects. AWS is robust but can be complex for newcomers.
In terms of configurations, consider at least 1GB of RAM for basic functionality, with the ability to scale as your app grows. For databases, PostgreSQL is a solid choice, especially with Django, due to its compatibility and performance.
  •  


If you like DNray forum, you can support it by - BTC: bc1qppjcl3c2cyjazy6lepmrv3fh6ke9mxs7zpfky0 , TRC20 and more...