Batch on Much Ado About IT

Cygwin BASH function to open the latest version of a document

Julian Knight — Wed, 15 Jul 2009 21:43:50 +0000

One handy function I’ve added to .bashrc (so it is always available) under Cygwin (the LINUX command environment for Windows) works out the current working version of a document. It assumes that you keep copies that have a version number or date in the file name that will sort correctly.

You can use it with an alias like this:

If you name your documents sensibly such as “a-document-2009-07-20.doc” or “a-document-v01.01.doc”, then the latest version of the file will be opened in the default application

Technorati : BASH Diigo Tag Search : BASH

Speeding up Cygwin

Julian Knight — Wed, 15 Jul 2009 14:54:00 +0000

Yesterday I mentioned my success with Cygwin. One issue I did have though was with the speed of startup. It was taking 15-20 seconds to start a BASH shell. It turns out that this was a PATH issue. I went through my Windows PATH and cleared out the clutter. Now it takes just around 3-4 seconds for a full BASH login and less still for just running a script. I now find myself using the BASH shell for all sorts of things and I’ve set up a number of alias’s to switch to folders I’m using a lot and to open common documents. One handy function I’ve added to .bashrc (so it is always available) works out the current working version of a document. It assumes that you keep copies that have a version number or date in the file name that will sort correctly. You can find the code on my development blog. Here are a few more alias’s I use:

Windows command prompt vs PowerShell vs Cygwin for remote backup scripts

Julian Knight — Mon, 13 Jul 2009 14:44:00 +0000

I’ve been struggling with trying to get a new backup routine working for my Laptop. I should point out that I have several complex requirements for backup so my needs are probably not average. However, it really shouldn’t be this hard! I need to use a combination of BZR (Bazaar) for document version control and RSYNC (for files that don’t need version control and for those folders that might contain files too big for version control systems – around 1/3 to 1/2 available memory). All of the backups need to happen over a secure link since I am often outside of my home network – indeed quite often behind locked-down enterprise firewalls but that’s a story for another time. So I use SSH (Secure SHell) to manage the secure connection and transmission. Thankfully both BZR and RSYNC can both use SSH as a transport. I don’t want to have to enter my remote system password loads of times though and this is where things started to get annoying. Using the Windows native versions of BZR, RSYNC and SSH I could not get a single shared password to work no matter what I tried. I also had some problems trying to control the output from the various tools and use it to further control what happens next – for example getting an IP address and working out whether I am on a network and where that network is. I tried to do this with a Windows command script first but even with the Windows 7 extensions it really is far to hard to get anything useful done and when I found myself turning to more and more utilities to help I thought “enough is enough”. At that point I happened to be reading an article on Windows PowerShell, the .NET scripting host so I thought I’d give it another go (having tried it before). I soon found that, although powerful for controlling the WMI interface, it is desperately convoluted and annoying for general use. So, realising that most of the tools I wanted to use have their roots in the UNIX world, it would make sense to try out the latest version of Cygwin. This has really come on a long way since it’s early days and is far more mature. It is also very much lighter in weight than the Microsoft provided UNIX services for Windows or whatever they are currently calling it. The Microsoft provided tools load perhaps a dozen services into memory permanently though they are rarely required. Cygwin only uses memory when it needs to. After converting my backup script from Windows batch to a BASH script under Cygwin, I soon had everything working as I wanted it – including the seemingly intractable problem of the shared passwords, now using KEYCHAIN to manage the SSH-AGENT and keys. So now I only need to supply a password once, it is held reasonably securely in memory and used by SSH as and when required. I only need to enter it once per reboot. The full script not only backs things up, it also auto-commits changes to BZR and changes network settings to match my current location and proxy server requirements. The later is not yet converted from batch as I don’t need it just at the moment. Let me know if you are interested in a copy of the script and I’ll upload it somewhere.

How-to show a Message of the Day (MOTD) at the Windows Command Prompt

Julian Knight — Tue, 12 May 2009 22:33:00 +0000

One of the features available under UNIX is the Message of the Day (MOTD). This is run every time you start a command prompt and displays the content of a file. In addition, the UNIX shells allow all sorts of stuff to be run and configured every time you start a new prompt using the .profile and .bashrc command files.

Windows users don’t generally expect that kind of flexibility from their command prompts. However, Windows does indeed support the use of an autorun into which you can shoe-horn any command you like.

So for my standard setup, I make the shell autorun run a .profile.cmd file that sits in the %USERPROFILE% folder. From that file, I can run anything I like.

To set up a shell autorun, you have to edit the registry so the usual warnings to be careful and back things up apply.

There are two locations you can set, one for the machine as a whole and one for the logged-in user.

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Command Processor
HKEY_CURRENT_USER\SOFTWARE\Microsoft\Command Processor

If you want to set an autorun for another user, you need to go down HKEY_USERS and find the appropriate one. It is really easier just to log in!

In one or both of those locations, add a new “String Value” (REG_SZ) called AutoRun with the value:

%USERPROFILE%\.profile.cmd

Now create that file and put in a message such as:

@echo "Hello and welcome to my command prompt"

Save the file and open a new shell and you should see the message just after the Microsoft copyright.

This should work on all versions of Windows at least from XP onwards.

If you want to add this to a batch file to set up new machines, here is the command you need:

reg.exe ADD "HKLM\SOFTWARE\Microsoft\Command Processor" /v AutoRun /t REG_SZ /d ^%USERPROFILE^%\.profile.cmd /f

(Note that the above needs to go on a single line)

Updates

2018-04-22 - Tidy formatting, reformat for Hugo output.

Technorati : Batch, Configuration, Scripting, Vista, Win7, Windows Diigo Tag Search : Batch, Configuration, Scripting, Vista, Win7, Windows

Keeping Control: File and Folder Links for Windows Users

Julian Knight — Mon, 27 Apr 2009 21:12:00 +0000

A good backup strategy for any computer involves keeping control of where stuff is stored. The fewer locations that contain files that change, the fewer locations have to be maintained. UNIX users have always had the ability to keep things wherever they wanted and then to LINK that information into the required location. Basically, links create a link or tunnel between one file or folder and another. Most of the time, you will not notice that you’ve entered a tunnel and you are not interested really. Windows users, however, have always been the poor cousins here. Stuck as we were in FATland, we had no access to fancy features such as links. So Microsoft in their inimitable fashion created a poor-man’s link – the Windows Shell Shortcut – so that the Windows GUI had some minimal capability (really only for menu’s and Windows Explorer). Windows 2000 improved on this by introducing “Reparse Points” one form of which is the “Junction“. This is an extension to NTFS that allows folders to be joined (linked) to another location in the local volume space. Making junctions is not an obvious process, you can do it from the disk manager and there is a tool in the Windows 2000 server resource kit called linkd. The POSIX tools included in the resource kit contain the UNIX command ln which can also create junction points and hard links; fsutil in XP can also. There are some third party tools too. It’s odd because I seem to remember that OS/2 had some kind of linking feature. Anyway, links of the UNIX type are a massively useful feature that has finally (with Vista, Windows 2008 and beyond) made it fully to NTFS and Windows. Vista, Windows 2008 and Windows 7 all have a command line tool called mklink. This can be used like the Unix ln command to create both hard links (which must be on the same volume) and soft links. Soft links under Windows can, in fact, span across SMB network drives as well. You might also like to look at another free tool called “Link Shell Extension” by Hermann Schinagl. This integrates into Windows Explorer, the web site also has a more complete explanation of the history of links in Windows. LSE does a number of clever things and is well worth a look. Hermann also has a “dupmerge” tool on his web site that will replace duplicated files with hard links. So now, if we want to tweak the HOSTS file for example (c:\windows\system32\drivers\etc\hosts), we don’t need to leave in place since that would mean that we would need an extra backup routine. Instead, copy it to somewhere that already gets backed up. Delete the original file and then from the command line:

Now you can edit the hosts file from either location, there is only one file (in %USERPROFILE%\BACKUPS). The difference being that even if you delete the file from its normal location, it will still exist in the “real” location. If you delete it from its “real” place in BACKUPS of course, the link will be broken and wont work. To link a complete folder, it is the same command with a /D parameter added. For example, I keep a folder of command line utilities such as ls, ssh and rsync in a folder on a USB pen drive. I sync that folder to the BACKUPS location on my hard drive for convenience but I need the folder in my PATH otherwise its hard to execute the utilities. I don’t want a really long path, it’s bad enough already, so I link the folder to c:\cmd with the following:

Now I add c:\cmd to the path and the utilities seem to be in both places. I’ve said in other posts that I like to reinstall Windows now and again but it can be a pain to restore all of the document files. Similarly, if you keep multiple operating systems on your hard drive, how do you keep your documents sorted? One way is to put all documents, videos, music, etc. onto a separate partition. Now, instead of going mad with the Windows registry trying to relocate your normal documents folders to another drive. Simply delete the normal documents folder – %USERPROFILE%\Documents\ under Windows 7 and relink it to the appropriate folder on the other drive as so:

Put this in a script that you run when you reinstall Windows and its easy and quick. One final note. You may find a few pieces of software that cannot cope with links. Certainly Subversion cannot though Bazaar can. Windows Explorer seems OK though as do utilities such as RSYNC.

Technorati : Backup, Batch, Configuration, Scripting, Vista, Win7, Windows Diigo Tag Search : Backup, Batch, Configuration, Scripting, Vista, Win7, Windows

Changing system environment variables from the Windows command line

Julian Knight — Tue, 24 Mar 2009 11:21:00 +0000

There are several ways to change global or user environment variables manually in Windows. Most are well known so I wont repeat them here (e.g. in Vista or Windows 7, Control Panel/User Accounts, Change my environment variables). However, sometimes you want to do this from a command (aka script or batch) file. This is not as straightforwards as it might seem. That’s because if you simply set the variable – e.g. set FRED=JimBob – it is only set while you are in that command file. Once the script has finished, the variable will no longer be set. There are a number of examples of setting system or user environment variables available if you do a Google search but most of them are incomplete – they do not immediately make the new value available to all applications (and particularly to new command shells). To make sure that the new value is available system-wide, you have to tell Windows to refresh the environment variable list and the easiest way to ensure this happens is to change the variable from a [Windows Scripting Host][1] ([WSH][2]) script. Here is an example script to do this. Save this file as something like set-env.vbs somewhere convenient.

Note that in Vista or above, you will need to run the command file with elevated privileges for this to work. The normal command prompt gets this automatically but if you want to run the file from Windows Explorer, you will need to create a shortcut and change the settings. You can now use this in your batch command scripts, for example:

Technorati : Batch, Configuration, Scripting, VBS, Windows Diigo Tag Search : Batch, Configuration, Scripting, VBS, Windows

How to get and use your local IP address in a Windows 7 (and Vista) batch command file

Julian Knight — Fri, 20 Mar 2009 14:11:00 +0000

If, like me, you spend a lot of time on a variety of customer sites, you will probably be familiar with the issues around swapping networks. I’ve already blogged about the problems with Windows 7, Vista and Firefox proxy settings and I will do some more articles on getting on with problematic proxies later. However, I wanted to let people know how to get hold of your IP address from within a batch (command) file.

Note the space after the : on line 3. The FOR command used twice here splits the text output from what is in the () after the “in” using the defined delimiters (“:” and space in the 1st case, “.” in the 2nd). In the first FOR statement, we take element number 13 only, it ends up in variable %%i. In the second case, we take elements number 1 to 4, they go into variables %%a, %%b, %%c, %%d. Now you have not only the full address but also the componants so if you wanted to check whether you were in a particular class C network, you could do something like:

That would check if your local IP address is between 10.97.100.1 and 10.97.199.254 Note that I’ve used the enhanced FOR statement – the “usebakq” makes the FOR statement more like a UNIX type one where commands are enclosed in back-quotes (`). This certainly works for Windows 7 and should, I think, work for Vista. Prior to Vista, you would need to do things differently anyway. At the very least you would have to search for “IP Address” in the FIND statement as IPCONFIG didn’t include IPv6 information. Now that you have everything in place, you can control the proxy settings for each application that needs access out of the local network. I’ll blog about that another time.

Technorati : Batch, Vista, Win7, Windows Diigo Tag Search : Batch, Vista, Win7, Windows

Shell script to Back up critical files (using RSYNC)

Julian Knight — Thu, 14 Aug 2008 01:57:00 +0000

Following up from my article on backing up USB drives, this recipe backs up the critical files on my desktop to remote storage (a NAS device on my network). Note that PC2 is the desktop to be backed up, SLUG1 (192.168.1.2) is the NAS device and USER1 is the user id doing the backup.

I have a similar script that runs on the NAS device which backs key files on that to a remote hosting service on a different continent! That way, I don’t have to worry about the house burning down or being burgled.

Automatically Backing up a USB Drive with RSYNC (KDE)

Julian Knight — Thu, 14 Aug 2008 01:22:00 +0000

USB Drives of all kinds need to be backed up and the best backup is an automatic one (it’s the only way to make sure that it gets done!).

So here is one recipe for doing just that using RSYNC and some BASH scripting magic.

I’ve split this into two files. You don’t have to do this of course and one may well be better for you. I used two because I can run the second one manually as well. Put everything in autorun.sh if you want to backup each drive individually, however, note that KDE produces an annoying extra dialog (a security warning) asking if you really want to run the autorun.

autorun.sh

This resides in the root of the USB drive and is executed automatically by KDE when the drive is detected (though not if the drive is attached when booting)
usb-backup-manual.sh

This is a bit of a nasty hack, I have manually configured a list of drives that might be attached so that I can back them all up together. Not elegant but it works for me.

autorun.sh

Note the use of KDialog to provide a minimal GUI. In the second file, KDialog produces a progress bar.

Also note the RSYNC parameters. These are always painful to get to grips with so it is nice to have an example to work from. In this case I am backing up so I am making sure that the backup is an exact copy of the original (as opposed to synchronising which would allow changes to happen on either side).