How to configure and run file-syncing macros in MobaXterm on Windows platform
Some MATLAB programmers prefer to develop codes on their own PC but run jobs on the SCC cluster. That requires upkeeping of multiple copies of the same file on the two platforms. Keeping these copies up-to-date requires effort and potentially create file sychonization issues which can lead to unintended consequences. Demonstrated below is a procedure to facilitate a more robust way to ensure files on the PC and the SCC are in sync.
The README file (you are reading it) explains and demonstrates the procedure for creating the necessary
Linux configuration file to facilitate file syncing between a folder on your PC with another on the SCC.
Note that while a MATLAB example is used for demonstration, the procedure in general is not limited to MATLAB.
Note also these notations throughout this page:
- /home/mobaxterm or [...] -> PC window prompt
- scc1$ SCC prompt
- >> MATLAB workspace prompt
MobaXterm basics
Before we go on to the main topic, let's review the following MobaXterm file structure
layout and naming. When you launch MobaX by clicking on the MobaX icon, the only window tab that appears, by default, is your /home/mobaxterm folder (the tab label indicates that).
Copy zip file from the SCC to your PC
- Find out where your local PC matlab folder path is in MobaXterm representation. It should have the general form of
/drives/c/windows/...
.
- On the SCC, decide where you want to copy your PC's matlab folder over. Make note of the path (e.g., /usr3/graduate/userID/... or /project/project-name/userID/...).
- Go to /home/mobaxterm
[...] -> cd
- Copy dotfiles.zip from the SCC to /home/mobaxterm
[...] -> scp -rp yourUserID@scc1.bu.edu:/project/scv/examples/matlab/misc/howto-sync-files/dotfiles.zip .
dotfiles.zip 100% 4195 3.6KB/s 00:00
- Unzip the
dotfiles.zip
file[...] -> unzip dotfiles.zip
Archive: dotfiles.zip
inflating: .aliases
inflating: .bash_profile
inflating: README.html
creating: bin/
inflating: bin/mysync
inflating: bin/rsync-test
- Run the
.bash_profile
[...] -> source .bash_profile
- Make sure that the following command shows the path for
mysync
[...] -> which mysync
/home/mobaxterm/bin/mysync
- Go to the ~/bin directory
[...] -> cd bin
- Use either MobaTextEditor (aliased to medit) or the
vi
editor to change the source and destination in the template mysync
with your own [...] -> medit mysync
Information on file syncing
- The underlining script employed is
rsync
- Recommend to place all your scripts in the ~/bin folder (for example,
mysync
). Files residing in ~/bin
can be conveniently executed from anywhere (on the same platform).
- There are multiple ways to define your application scripts depending on the objective. If your goal is to always deal with a fixed PC and a fixed SCC folder, it is more convenient to hardwire the PC-SCC folder names into a specific script (such as mysync) to minimize typing and avoid potential typographical errors. On the oposite spectrum, the script syncAB expects you to provide the PC and SCC folder paths and names every time you run it.
- The "-ra" options perform task recursively and preserves the original time stamp.
- The "--update" switch is to update only files between PC and SCC that are found to have different time stamp and sizes.
Unchanged files will not be transferred. As a result, the syncing process is more efficient if only a handful of files needed be updated.
- To sync from the SCC to the PC, just reverse the order of the source and destination. Note that unless your PC has an internet address, the command must originate from the PC window.