Local

The goal is to safely synchronize files locally, create a local copy from the remote for testing or debugging.

1. Overview

When working with a any web application hosted on a remote server, developers often need a local copy of the site for testing or debugging. Using rsync over SSH is a convenient way to do this because:

• Incremental Updates: Only changes are transferred after the initial sync.
• Preserved File Attributes: Permissions, timestamps, and symlinks can be maintained.
• Secure: Data travels over SSH, so passwords and data are encrypted.

In this guide, we assume you already have SSH access to the remote server.

2. Prerequisites

1. SSH Access

   • You should be able to SSH into the remote server without issues. Either by:
     • SSH keys (recommended), or
     • Password-based SSH authentication (less secure).

2. Installed Tools

   • A terminal (e.g., macOS Terminal, Linux shell, Git Bash on Windows).
   • ssh and rsync installed (common on Linux & macOS; on Windows, install via packages like Git for Windows or WSL).

3. Remote Server Login Details

   • Hostname or IP (e.g., my-website.com or 192.0.2.10).
   • Username (e.g., ubuntu, root, admin).
   • Remote Path to the website files (e.g., /var/www/my_site).

4. Local Directory

   • A folder on your machine where you want the website files (e.g., ~/dev/my_site_local).

3. Basic Rsync Command

A typical rsync command to pull (copy from remote to local) looks like this:

rsync -avz --progress user@host:/remote/path/ /local/path/

Here’s what each flag does:

• -a (archive): Preserves file permissions, ownership, timestamps, symlinks.
• -v (verbose): Prints detailed output so you see what’s happening.
• -z (compress): Compresses data during transfer to speed things up (especially useful over slower connections).

Example:

rsync -avz --progress myuser@my-website.com:/var/www/my_site/ ~/dev/my_site_local/

Note: Ensure you have the trailing / after /var/www/my_site/ so rsync copies the contents of that directory into my_site_local rather than nesting them under another folder.

4. Excluding Unnecessary Files

Often, you might exclude certain large or irrelevant files—like cache directories, logs, or user uploads not needed for local dev.

rsync -avz \
  --exclude='cache/' \
  --exclude='*.log' \
  myuser@my-website.com:/var/www/my_site/ \
  ~/dev/my_site_local/

Common exclusions:

• cache/ directories
• .git/ or .svn/ version-control folders
• *.log or *.tmp files
• node_modules/ Exclude Node.js packages (you can re-install locally)
• vendor/ Exclude Composer packages (you can re-install locally)

5. Keeping Local in Sync

After the initial pull, you can re-run the same rsync command at any time to grab updated files. Because rsync is incremental, it will only transfer changes—making subsequent pulls much faster.

If you want rsync to delete local files that are removed on the server, add --delete. For example:

rsync -avz --delete user@host:/remote/path/ /local/path/

Use with caution; it will mirror the remote server exactly, removing files locally that no longer exist on the server.

6. Example Workflow

Below is a typical local dev workflow:

1. Clone or Create Local Project Folder

   mkdir -p ~/dev/my_site_local
   cd ~/dev/my_site_local

2. Pull Files from Remote

   rsync -avz --exclude='cache/' myuser@my-website.com:/var/www/my_site/ ./

   • You now have a local copy of the remote website.
3. Database Sync (Optional)
   • If your site uses a database (e.g., MySQL), you might also dump the DB from the server and import it locally:

     # On remote server (or via ssh):
     mysqldump -u db_user -p db_name > /tmp/db_name.sql
     
     # Then copy that dump locally (SCP or rsync)
     scp myuser@my-website.com:/tmp/db_name.sql ~/dev/my_site_local/
     
     # On your local machine:
     mysql -u local_db_user -p local_db_name < ~/dev/my_site_local/db_name.sql

4. Local Development
   • Start your local server or environment to test the site.
   • Make changes, fix bugs, etc.
5. Re-run rsync
   • Whenever you need a fresh copy, just run the same command again.

7. Common Troubleshooting

1. Permission Denied / SSH Keys
   • Confirm your key is correct, or if you’re using a password, ensure you provide it when prompted.
   • Check file/folder permissions on the server—some directories may be restricted.
2. Host Not Found
   • Verify you have the correct domain/IP address. Check your network connection.
   • Try ssh user@host to confirm you can log in before running rsync.
3. Large Files
   • If transferring a lot of data, a single run might take a while.
   • Consider rsync --partial to resume partially transferred files if the connection drops.
4. Excessive Local or Remote Files
   • Use the --exclude option or incremental approach to avoid transferring unneeded large files.

8. Example Script for Repeated Use

If you frequently need to grab updates, you could create a small shell script:

#!/usr/bin/env bash
# sync_site.sh

REMOTE_USER="myuser"
REMOTE_HOST="my-website.com"
REMOTE_PATH="/var/www/my_site/"
LOCAL_PATH="$HOME/dev/my_site_local"

EXCLUDES=(
  "--exclude=cache/"
  "--exclude=*.log"
)

rsync -avz "${EXCLUDES[@]}" \
  $REMOTE_USER@$REMOTE_HOST:$REMOTE_PATH \
  $LOCAL_PATH

1. Make it executable:

   chmod +x sync_site.sh

2. Run anytime to refresh:

   ./sync_site.sh

Syncing a remote site to your local environment with rsync ensures you always have up-to-date files for development or debugging. Combining rsync with SSH keys and optional excludes, you get a secure, efficient, and repeatable workflow.

• Keep your local environment in sync without manual file copying.
• Leverage excludes to avoid unneeded files (like caches or logs).
• Incrementally update your local copy whenever changes occur on the server.

Whether you’re troubleshooting a bug, reviewing logs, or working on new features, this method drastically simplifies local site management. If you have additional needs—like syncing databases or ignoring massive media folders—simply extend these basic rsync commands to fit your workflow.