Checklist of troubles that can happen when moving data center

Started by nisha03, Jul 29, 2022, 10:36 AM

Previous topic - Next topic

nisha03Topic starter


The process of moving is full of routine and works in much the same way in companies in our industry. I will not get hung up on issues such as choosing a contractor for transportation or loading and unloading (although they are also important!).

I devoted most of the article to all sorts of difficulties that await when planning a move or installing equipment.




Training

It is important to calculate all things in advance in order to avoid stupid mistakes when moving.

    Rehearse. Always try to assemble apparatus at your website first before taking it to a new one. That's what we did, and it saved us from surprises. All apparatus  was rack-mounted and labeled in New York. Crossing logs and layout were created. Why - read below. After that, the apparatus was dismantled and packed in special shipping boxes according to the size of the carrier's car body, which guarantee the safety of the equipment during transportation.

    Get acquainted. Familiarize yourself with the new place in advance, it is best to visit it in person, look around and chat with the staff. Get samples of all the documents that you will need to provide to enter and open applications in the helpdesk.
Find out the rules for unloading: where is the ramp, how to drive up to it, how to call an engineer to accompany you, at what time unloading work is possible. Check warehouse hours! We have found ourselves in a circumstance where fitting in the data center is possible 24 hours a day, and all apparatus is placed in a warehouse in advance. But the warehouse worked from 10 to 18-30 :)
    Do the math. Before going, prepare a comprehensive inventory of the apparatus being transported, literally from the rack to the nut. You must have all models, serial numbers, etc. rewritten.

    Label. Make a crossover log, mark all apparatus (at least on both sides) so that you don't panic in a new place, what's what. Measure your rack, including its useful size (depth, width, height) - you will see why this is important a little later. Carry out markings according to a single standard. Be sure to mark all ports on the equipment with a permanent marker.

    Write it down. Coordinate all documents and powers of attorney that you will need to take with you. The power of attorney must be issued in the form of the host organization. Do not rely on employees, check and double-check their work. Even a small mistake can ruin your deadlines.
    Get yourself together. Take an alarm case with you: a laptop with a energy supply, screwdrivers (straight, Phillips, short Phillips, powerful), pliers, stationery knife, scissors, Velcro for fixing wires, cable ties (not for wires!), marker, energy adapter C14 to euro , tested USB-COM cable, flash drive, APC rack key, notepad, pencil, marking printer or labels for marking, enough sheets for writing. Contacts of your colleagues and contractors on paper and the ability to call them not by cellular communication (it may not be in the data center). Just take my word for it!


Measure seven times... And measure again!

Despite all the measures taken, incidents do happen when going. I'll tell you about the most typical ones.

     Too long cables. The delivery included 10 m sas cables, 4 per rack. This length was remarkably enough for us in New York data centers, but in St. Petersburg the wire tray turned out to be lower, and the length of the cables turned out to be excessive. The problem is small, we found a way to neatly lay the wires.


New configuration. The 200-kilogram 3U top storage shelf, which was perfectly located in the 20th unit on the apparatus diagram, unexpectedly :) had to be pushed forward to its full length, for which I had to remount it down. This means that the storage controller needs to be moved down, under the shelf, to provide capacity expansion. This entailed redesigning the installation of other apparatus . No one is immune from such circumstance, it is important to be able to solve them on the fly. Therefore, entrust the move to the most experienced engineers.

Power connectors. Here is another example: among other items, we brought Power Distributor Units (PDUs) - energy modules to which servers are connected. These modules are available in single or three phase and require single or three phase power respectively. There are also "pears", remote control connectors, they are also different: 16 and 32 amperes. You request a 3-phase energy supply, it will be organized according to the application.

And now, it would seem: there is a three-phase plug, in front of you is a three-phase socket. But they do not connect, because the plug is 16 amps, and the socket is 32. The difference between them is literally half a millimeter, not visible to the eye. Even before the move, we clarified this point, took what we needed, but for safety, we were still equipped with mating parts of the connectors.
And measure again. Some overlays appear quite unexpectedly: we brought to the website apparatus with a depth of 980 mm for a 1000 mm rack, and its actual fitting depth did not match the nominal one - it turned out that the working space in it was 750 mm. And in the 1200 mm rack, after installing the 970 mm equipment, there were 3 free centimeters instead of 23. Again, be sure to measure everything that you install and transport.

Attachment point. Sometimes our engineers have to be creative on the spot. If you have worked with standard APC racks before, then you probably know that there are special holes in them to place the remote control, where the remote control snaps into place remarkably. And if the rack is made by another vendor and there are no holes for the remote control, you can get into trouble: do not pick it with a drill, right word! Fortunately, the engineers took with them ordinary metal corners, which turned out to be neat and reliable fasteners.


Attention, oversized! Another potential issue is that the pear power connectors may suddenly appear not under the raised floor, but inside the racks, in the "paid" zone. That is, they occupy the places of units for how much in vain. Let's say it was planned to place 42 units in the rack, but in fact 2 units are already occupied by "pears".
And you have to place the minimum overall apparatus below, which has no communications behind. Such circumstances can break the entire circuit of the apparatus. In our case, the issue was solved by placing boxes for spare parts at the bottom of the rack, which have neither electrics nor other wires in the back.


Just in case: there is nothing supernatural in this process, and the apparatus does not suffer - the discrepancy in width is slightly less than a notebook cell.

Stock up

Another item that seems obvious and is therefore always forgotten: stock up on fasteners. Even if it seems like it should be enough, take more. Looking for or buying them on the spot is more expensive for yourself.

We needed to mount the equipment on site in two racks. We took with us three sets of 50 nuts and bolts, which, according to the sensations, should have been enough with the head.
But that was not the case: in this configuration, apparatus, plugs and decorative panels are fastened with nuts and crаckers, and not with latches, as we are used to. As a result, in order to secure everything firmly, I had to buy another set, which was enough almost back to back.

Approximately the same applies to wires: the more choices at the website , the better, bring the extra ones home. Take the maximum: energy , network cables, and Ethernet of different lengths. The tray you pass wires through can hang at varying heights above the rack, and if you have at least seven meters of cables, you'll end up with a tangle of wires inside or on top of the rack that you can't work with. You need a guarantee that you will be able to lay all things nicely and neatly and the server room will not resemble a snake's nest during the mating season.

    Life hаck:
    Be sure to take with you a master who knows how to lay "beautifully". According to rumors, this is a superpower that only a few are perfect at.


And again about labeling

The installation of the rack is completed, not a single engineer was injured. But that's not all. Ideally, after fitting, you need to draw a detailed layout of the apparatus. At the same time, "portraits" are made both in front and behind. Each is labeled with a rack number. This is necessary so as not to confuse what is on the left and what is on the right.

Remember the pretty wires? In addition to aesthetics, they are useful during operation and reduce the risk of error. For example, you need to reconnect only one wire, and four more stretch from the ball. As a result, important services will be disabled.

Therefore, even if you are very lazy and all things seems (to you now) simple and understandable, be sure to mark all wires and all apparatus. At least with a marker and paper tape. The engineers who will support you after the move will thank you.

It is even better if the wires in critical nodes have not only signatures, but also a different color. Suppose an engineer is given the task of disconnecting a particular black wire. All wires are black. What is the probability of an error? ... We use green wires for the left PSUs and orange wires for the right ones. Power supplies from different sides are connected to different energy beams.


Rules of behavior

Finally, let me remind you: all data centers have their own rules of conduct, and your employees should know them before starting work.

Be sure to check with the attendant in advance about the following points:

    Where can / can not go, how does the access control system work in general?
    How to rapidly go outside?
    Where to throw garbage?
    How to take back the remaining parts?
    Where are the restrooms?
    Where is water and food located (canteen, vending machines, etc.)?
    Where can I get fresh shoe covers?
    How to quickly contact the duty officer?
    Will it be probable to leave the hall on my own, without an attendant?
    Where is the console with the monitor?
    Is there internet access via Wi-Fi? Get a login and password.
    Is it possible to do interstitial cross-country crossings?


And a couple of special items to know about in advance:

    Be sure to find out until what time the passes are valid. Engineers regularly stay at the facility after midnight. The security system can reset the passes at 00:00 and people will not be able to get out.
    When installing apparatus, an electrical safety permit may be required. For this work, you must have the 2nd group of admission, and without a state-issued certificate, you will not be allowed into the data center.


If all points are checked and there are no issues, you can proceed with fitting at the facility.


This is the checklist I came up with. No move is 100% smooth, but following these steps will save you a few nerves.
That's all, ask questions and share your experience in the comments.
  •  

berto

One more non-obvious point when moving from your "server" to a real data center: equipment with one power supply (firewalls, switches, tape drives) needs an ATS, since the power beams are regularly turned off for maintenance.
At the same time, getting your own will be beaten off in a couple of months compared to renting, and it can be put in one unit with half-depth switches.
  •  

Nizam18

When executing each new project, we not only rely on the general scheme of work, but also take into account the individual requirements of the customer company.
These may be individual features of the new office site and its infrastructure, characteristics of equipment and software, dates and hours of work.
For example, stopping business for the time of moving is highly undesirable for some companies, so we are asked to carry out work exclusively at night or on weekends. To transfer productive loads during physical transportation, we create demo stands.

Of course, we meet them halfway, because our main goal is the comfort of the client. We try to ensure as much as possible the continuity of all key DC processes of the client's company during the relocation.

As part of that service, we provide secure transfer of data center network infrastructure, processing and storage infrastructure, as well as virtual infrastructure. In addition, our services include the transfer of UPS, air conditioning systems and even gas fire extinguishing systems. In each project, we consider the customer's situation individually.
It is especially important that the customer's data center is launched on a new site without interrupting client services, which ensures that there are no risks and losses.

  •