#YOLO Programming

This isn't necessarily developer-centric, but any developer that backs up their files to a Seagate Central NAS may appreciate this because of the thousands of tiny files we face in our day-to-day.

Note that this is our first blog post at yoloprogramming.  The style that we've decided on (for now at least) is to write as if we went on a journey; what we tried, where we failed, and what and learned along the way.  If you want to get to specifically to what I think is the easiest/fastest approach, scroll toward the end of this post.

Further, since this post is not relevant to only developers, I spell out the steps so that non-IT professionals can also follow along.

The Journey

My situation was as follows, in case anyone reading this is trying to rectify the same problem: I'd been using the Seagate Dashboard application that came with the device in order to make incremental backups.  Over the years I've stopped and started different backups for whatever reason* (that is, stop the existing incremental backup and create a new one). In it's wake the app left hundreds of thousands of files, mostly tiny.  Attempting to delete from within Windows was a nightmare, it would take hours and hours.  To add insult to injury, I'd also run into "file path too long" errors, which would stop the deletion.

Option 1 - Map the backup path to a drive and use Windows' delete utilities to delete the backup (performance with this approach may be good enough for you)

1. To overcome the "path too long" errors, I mapped the Seagate Central backup path to a drive letter. In my case, the network drive was already mapped to z:, so at the command prompt (for non-developers, hit your windows key and r at the same time, and type "cmd" without the quotes) .  At the command prompt, type 

   subst x: "Z:\Seagate Dashboard 2.0\VEYRON\John\Backup"

   Substituting the appropriate drive letters and file paths for your situation, of course.  When this is complete the x: drive will be mapped to that network file path.  For completeness's sake, if you want to remove the mapping, type the following:

   subst x: /D

    

2. Now to discover the fastest way to delete on Windows.  A Google search landed me onto Stack Overflow, specifically this answer. 

   The best I've found is a two line batch file with a first pass to delete files and outputs to nul to avoid the overhead of writing to screen for every singe file. A second pass then cleans up the remaining directory structure:

   del /f/s/q foldername > nul
   rmdir /s/q foldername

   So I dutifully created "delete.bat" on our newly created x:\ drive, with the following contents 

   del /f/s/q 4eafcd93-45c9-441f-85fa-9ec1cc53a48f > nul
   rmdir /s/q 4eafcd93-45c9-441f-85fa-9ec1cc53a48f

   where 4eafcd93-45c9-441f-85fa-9ec1cc53a48f is the file path Seagate Central decided to make for my backup.  Yours will clearly be different. 

3. Finally, I ran the batch file and waited.  And waited.  Being impatient, I canceled the batch (for non devs, hit ctrl and c at the same time at the command prompt) and edited the batch file by removing "> nul" (this was originally added in to make the batch faster).  This allows me to get feedback every time a file is deleted.  So I re-ran the batch file and the deletes started to drip in.  Waaaay too slow for my needs.  That said, if you have less files than me, or more patience, this approach will be good enough and doesn't require any special tools.

Option 2 - Use "rm -rf" from Linux or Mac (performance with this approach may be same as Option 3)

From my understanding, the Windows implementation for delete is less than ideal, so it had been suggested that deleting from *nix (for our purposes Linux or Mac) would be faster.  So I booted up Ubuntu in a virtual machine (booting Linux in a VM is beyond the scope of this post, I encourage you to Google it if you're interested) to get access to a Bash terminal.  If you have a Mac available then you already have Bash. Developers may be wondering why I didn't use the Linux Subsystem for Windows.  Well, I tried; but I was unable to mount the NAS.  Some remedial Googling indicated this was not possible at this time, so I gave up and booted the VM.

1. In Ubuntu, I browsed the the appropriate path on the NAS (which will be listed in the Network section of the file browser) and then right clicked to open a terminal window, where I could type the commands listed below.  In all honesty, from here I probably could have deleted the directory from the UI, which will likely do the same thing**.
2. At the terminal (where I had already navigated to  Seagate Dashboard 2.0/VEYRON/John/Backup) I typed 

   rm -rf 4eafcd93-45c9-441f-85fa-9ec1cc53a48f

   In all honesty, this may be as fast as the approach I'll discuss below.  I'm simply just unsure of how the rm command was implemented (is it chatty; does every delete communicate back to the caller?).  A chatty implementation would waste a lot of time on the network (My hunch is that it's not chatty, for what it's worth).  Note that I am not a Unix or Linux expert, I primarily work on Windows, so maybe the answer is common knowledge for people in that circle.  Regardless, since I was already on Linux, it meant I had ssh, so the following option didn't take much effort to try.  The following also makes life easier for those who don't have access to a Linux VM or a Mac; they can simply install PuTTY (an ssh client for Windows); so even if the performance is the same, there is less level of effort.

Option 3 - SSH into the NAS and delete from there

1. At the terminal type ssh [username]@[network address].  In my case that was [email protected].  You'll soon be be prompted for a password.  Use the same password that you used when you created your Seagate Central account.  If your backup is in your user folder, then you can simply navigate to the directory with your backup and run "rm -rf 4eafcd93-45c9-441f-85fa-9ec1cc53a48f" (replacing the backup path with yours, of course) to delete it. 
2. If your back up is not in your personal directory but in "Public", then navigate to the appropriate path.  To get to Public type

   cd ../Public

   Once you've navigated to the backup path (in my example the full path was /Data/Public/Seagate Dashboard 2.0/Veyron/John/Backup) you'll need elevated privileges to delete the files.  Easy-peazy just type

   sudo rm -rf [backup path]

   to begin deleting.

Note: While Options 2 and 3 are faster (much faster in my case) than option 1, if you have a lot of files it can still be slow.  In my case it took ~1.5h or so.  But compared to my initial attempt on Windows (deleting from Windows Explorer) it was night and day.

Hopefully you found this useful, or at least learned something.  I know I did!

_{*Usually an OS re-install; that is, I still had the original files on my data drive, but because I reinstalled the OS I wanted to start the backup from scratch.}

_{**It looks like the command line is faster than the GUI, according to this post}