-
Notifications
You must be signed in to change notification settings - Fork 6
Examples
This example shows how to
- Read a command line string
- Understand how a process communicates through a pipe
- Use the
walkandxoprograms - Ask those programs for help
Imagine the following problem: You need to some set of DLL files on a Windows system. The locations of interest reside in any top level directory on the C: drive matching the string "Prog". Particular examples are:
- Program Files
- Program Files (x86)
- Program Data
The solution is immediately presented:
walk -d -t 0 C:\ | xo -i Prog | walk -f - | xo \.dll\n
Above are four commands interconnected with vertical pipes. After the command line is inserted into a command interpreter such as BASH or cmd, each command becomes a processes that communicates through an anonymous pipe represented by the vertical pipe character. Processes are born from right to left, so starting from the right-hand-side, the standard output of the processes to the left of the pipe is connected to the standard input to the processes to the right of the pipe. This is done until the left-most process is reached.
Although processes on a command line start from right-to-left, data conveys from left to right across corresponding pipes. The process generating the data is usually on the left of the first pipe, it acts as the data source.
walk -d -t 1 C:\ | xo -i Prog | walk -f - | xo .dll\n
Walk traverses a list of files. In the selection above, this list is just one item: the root of the Windows C:\ drive. Using walk -d will print only directories, and walk -t 0 limits traversal depth to level 0, the starting level. The result: walk prints dirs in C:\ to stdout. Stdout is connected to the first pipe, conveying the data to the next process.
walk -d -t 1 C:\ | xo -i Prog | walk -f - | xo .dll\n
As you can infer from the above selection, xo is similar to UNIX grep or windows find programs. It searches for the string Prog in the input structure. Assume for this example that a structure is just a line of text.
When xo finds the substring Prog, it prints the entire structure (line) to stdout, which is conveyed to the second pipe.
walk -d -t 1 C:\ | xo -i Prog | walk -f - | xo .dll\n
The dash appended to walk -f - makes walk treat stdin as a list of directories to walk, giving you the ability to exclude or include directories matching certain expressions in the walk. When the dash is used as a file name in the file list, walk reads from stdin line by line, and for each line it reads, it treats that line as a named directory to traverse.
The result so far is that we enumerated three directories in the root of C:\ that were relevant to our interests and then piped them back to walk so it could enumerate the contents of those directories for us.
The reader may question why two walks are necessary instead of just one. The answer is that xo and walk do separate things. The walk program walks directories, it doesn't know how to filter directories by regexp. Xo doesn't know how to traverse a directory: it's just a filter. But the two programs can be combined elegantly to avoid duplication of functionality. I recommend looking at GNU find program for a good bad example, It bundles two functionalities using a chaotic set of command line flags. Instead of doing such silly things, we use independent programs with one role that communicate the result of their work to the next one.
The last command is xo dll\n, which searches for files ending with a dll suffix. As a result, the final output we see on the terminal is the output that makes it out of the final xo process: the list of DLL files inside the three directories.
At this time, we have defined a command line that composes communicating process. The output is the list of all DLL file names or paths residing in directories named "Proc" in the root of C:
walk -d -t 1 C:\ | xo -i Prog | walk -f - | xo \.dll\n
The next step is to generate a list of all these files along with the corresponding sha1 hash of their contents. This will allows us to store a list of these files on the file system and compare them to another list generated in the future. There are now several forks in the road, we can:
- Download sha1sum for Windows
- Write a batch file or shell script
select the file from the file listrun a sha1sum digest on the file contentspaste that sha1 digest alongside the file's name in the file list- repeat this for every file
However, the steps above are slow, daunting, and platform specific. The Torgo way is not.
walk -d -t 1 C:\ | xo -i Prog | walk -f - | xo \.dll\n | hash sha1 -
The first four commands are as they were before. The hash sha1 - command reads the list of DLLs, opens each one, hashes its contents, and outputs a list of tab separated digests and file names. The final task is comparing the implementation complexity of the two methods, left as a exercise to the reader.
The documentation for Torgo programs reside in the programs themselves. They can be accessed by asking the programs (walk, xo, hash, ...) for help using the -h flag or the -? flag. Some Windows users will expect /? to show the help too. Expect the unexpected: slashes are not argument separators, they are path separators along with the backslash character.