Tag Archives: shell

Weekend’s piece of shell magic

shell, ,

A few days ago I wrote a post about setting up squid as an anonymous proxy using multiple ips.

That setup would basically make squid listen for connections on certain ips and will create outgoing connections ( for fetching the requested page ) from the same ip that received the connection. The setup, as described in my previous post involves creating one acl and specifying one tcp_outgoing_address for each ip you want to use. Now that's ok if you have just a few ips or even a full class (because you can create one acl to match the whole class ) , but what do you do if you have multiple classes and in each class non consecutive ips? And I'm talking about many such ips not just 10, but 100 or more...

The shell Magic

Well if you have already defined the ips on your interface(s) and you want to use all those ips in squid then you can use this simple shell script that will parse the output of ifconfig and output the acls and tcp_outgoing_address directives for each ip.

  1. span style="color: #ff0000;">"inet addr"' '"acl in_$an myip $i""tcp_outgoing_address $i in_$an"

Run this script on the server where you want to install squid and it will output the acls and directives needed for using all the ips ( except 127.0.0.1 ) . then just copy and paste them into squid.conf.

Now shell magic is nice but if you want to get your hands dirty you might want to look into patching squid to do the same thing. With the squid outoing ip patch you will not have to write any acl or tcp_ougoing_address for it, but of course you'll have to do the "get source - patch - compile" stuff .. which some may find harder and others more fun 🙂

extracting fields in shell

shell, , ,

A lot of shell scripts require processing some kind of data structured in fields or columns separated by special characters ( space, coma, semi colon, etc... )

This is a short tutorial that shows you how you can extract the fields in a stream of data. There are several ways of doing this and each has it's advantages of disadvantages.

Here is what I use:

  1. Using cut

    The 'cut' program will allow you to extract the fields separated by one character. you can specify which field to extract, and what is the field separator.
    Example: echo "a:b:c" | cut -f2 -d':' will output b
    The cut program has the advantage that it is simple to use, almost ( all ) Unix flavors have it included in the base distribution and is relatively lightweight ( ~33Kb with no library dependency other then libc on my gentoo Linux )
    The problem with cut is that the field separator can only be a single character.

  2. Using awk

    awk is a pattern scanning and processing language somehow similar perl. Actually it is believed that perl was inspired by languages like awk, perl, C, and some others. Awk is a lot more flexible then cur and can do a lot more. You can actually specify a regular expression for the field separator.
    Here is an example for extracting the fields separated by one or more spaces:
    echo "a b c"|awk '{print $2}' - this will print the second field. As you can see I have not specified any separator because awk uses <space> as the default separator. <space> means any number of spaces here.
    You can specify a different field separator by using the -F parameter.

  3. Using a shell function

    this may be the simplest and fastest solution but will only work if the field separator is composed of spaces or tabs only. As you may know the parameters are passed to a shell function separated by spaces. so you can just make a function that has the sole purpose of returning the field ( parameter ) you want.
    If I want to get the third field from a line I would do a function like this

    1.  

    getfield a b ccc ddd would display 'ccc' . This is more useful in a script where you need to get a field value from a variable containing some text but not so mush with whole files.

Do you know any other/better method ? Feel free to share them in the comments

mysql replication monitor

databases, shell, , ,

This is not a tutorial about how to set up mysql replication. You can find all the details about how to set up mysql replication in the official mysql documentation. This is just a script that can be used to monitor a MySQL replication setup. A MySQL replication setup consists of a master server and a slave server. On the slave server there are two threads that run continuously, one is the I/O thread that fetches changes that occurred on the master server and one is the SQL thread that tries to run the queries that were executed on the master server.

Continue reading mysql replication monitor →