If you like DNray Forum, you can support it by - BTC: bc1qppjcl3c2cyjazy6lepmrv3fh6ke9mxs7zpfky0 , TRC20 and more...

 

Relocating Files Between Two Directories

Started by serviceuncle, Oct 30, 2024, 01:19 AM

Previous topic - Next topic

serviceuncleTopic starter

There are two main folders in question:
/var/data1/
/var/data2/
The data1 folder is organized in a specific way:
<version>/<service_name>/<build_id>/*<operating_system><architecture>.(zip|7z)
For instance:
3.217.100/alfa/32/alfa_linux_x86_64.zip
3.217.100/zero/32/zero-ctrl_linux_i686.7z

The task is to transfer this content to the data2 folder, while restructuring it so that the top level reflects the service name of the distribution. Additionally, you need to categorize the directories based on architecture:
alfa/3.217.100/32/linux_x86_64/alfa.zip
zero/3.217.100/32/linux_i686/zero-ctrl.7z

a. This should be done with a physical move of the files.
b. Alternatively, it can be done without physically moving them. During the transfer, ensure that 7z files are converted to zip format.
  •  


luked-c1964

Here's an example of a script that could be used to accomplish the task:

#!/bin/bash

# Source directory
SRC_DIR=/var/data1/

# Destination directory
DST_DIR=/var/data2/

# Iterate through the source directory
for version in $(find "$SRC_DIR" -mindepth 1 -maxdepth 1 -type d -print -name '*'); do
  version_name=$(basename "$version")
  for service in $(find "$version" -mindepth 1 -maxdepth 1 -type d -print -name '*'); do
    service_name=$(basename "$service")
    for build in $(find "$service" -mindepth 1 -maxdepth 1 -type d -print -name '*'); do
      build_id=$(basename "$build")
      for file in $(find "$build" -type f -print -name '*'); do
        file_name=$(basename "$file")
        file_ext=${file_name##*.}
        os_arch=${file_name%.*}
        os=${os_arch%_*}
        arch=${os_arch#*_}
        if [ "$file_ext" = "7z" ]; then
          # Convert 7z to zip
          zip_file_name=${file_name%.7z}.zip
          7z x -o"$DST_DIR/$service_name/$version_name/$build_id/$os/$arch" "$file"
          zip -r "$DST_DIR/$service_name/$version_name/$build_id/$os/$arch/$zip_file_name".
          rm -rf "$DST_DIR/$service_name/$version_name/$build_id/$os/$arch/*"
        else
          # Move zip file
          mkdir -p "$DST_DIR/$service_name/$version_name/$build_id/$os/$arch"
          mv "$file" "$DST_DIR/$service_name/$version_name/$build_id/$os/$arch/$file_name"
        fi
      done
    done
  done
done

This script uses find to iterate through the source directory and identify the version, service name, build ID, operating system, and architecture for each file. It then uses mv and 7z commands to move and convert the files accordingly. Note that this script assumes that the source directory has the structure you specified, and that the destination directory does not exist or is empty.
  •  

Firladon

The script leverages parameter expansion to extract relevant information from file paths, utilizing techniques such as pattern matching with grep -Po to isolate specific substrings. It also employs parameter substitution to remove trailing slashes and extract file extensions.

#!/bin/bash

# Define input parameters
main_dir=$1
target_dir=$2

# Extract version directory from main directory path
version_dir=$(dirname "$main_dir")

# Iterate through files in main directory
for i in "$main_dir"/*; do
  # Extract file name and build number
  name_service=$(basename "$i")
  build_number=$(ls -1 "$i")

  # Extract OS and revision information from file name
  os=$(ls -1 "$i/$build_number")
  revision=$(echo "$os" | grep -Po '(?<=_)[^\.]+')
  extension=$(echo "$os" | grep -Po '[^.]+$')
  os_name=$(echo "$os" | grep -Po '[^_]+')

  # Construct new directory path
  new_path="$target_dir/$name_service/$version_dir/$build_number/$revision"

  # Create new directory and copy file
  mkdir -p "$new_path"
  cp "$i/$build_number/$os" "$new_path/$os_name.$extension"
done

In the context of DevOps, this script can be utilized as a building block for more complex automation tasks, such as continuous integration and deployment pipelines. By leveraging Bash scripting and parameter expansion, you can create efficient and scalable solutions for managing complex directory structures and file systems.
  •  

mageba

In the realm of file handling, the initial 2-4 bytes typically house the MIME type. Yet, a significant hurdle arises with text files: how can we ensure that users initiate with the appropriate characters? Unlike scripts, which kick off with a shebang, LaTeX dоcuments might begin with comments or even blank lines. I've criticized gcc for this, but the reality is that to differentiate C files from other formats, a thorough examination of the entire file is necessary.

Unfortunately, some less experienced developers still depend on file extensions in certain situations. There's a clear opportunity here for collaboration to enhance file utilities, enabling them to analyze and categorize various text formats - be it markdown, plain text, LaTeX, or C source code. This would streamline workflows and minimize errors in file identification, ultimately improving the developer experience.
  •  


If you like DNray forum, you can support it by - BTC: bc1qppjcl3c2cyjazy6lepmrv3fh6ke9mxs7zpfky0 , TRC20 and more...