Monthly Archives: March 2009

This week on twitter 2009-03-29

  • wow, I just realized I have not tweeted in almost 5 days. Been on the road and now in switzerland @profdavidcosta #
  • In Zurich, saw a bit of the twon this evening, bought some watches and lots of chocolate, tomorrow I'll be leaving #
  • Finally back home, Zurich was nice. Looking forward to go back sometime. Maybe I'll have a chance to see more of the town next time #
  • what's the fastest way to read 264 posts in your feed reader? "mark all as read" :)#

Powered by Twitter Tools.

This week on twitter 2009-03-22

Powered by Twitter Tools.

This week on twitter 2009-03-15

Powered by Twitter Tools.

SSH to multiple servers and run commands

Problem

You need to run a list of commands on a list of servers and record the output of each command.

Solution

Create a perl script using Net::SSH::Perl ( a ssh client written as a perl module ). This script will read a list of commands from a file, a list of servers form another file, will connect to each server, execute each command in in the commands file then go to the next server and do the same.

Installation

Download the script: SSH Batch-0.1 (1.5 KB)
install Net::SSH::Perl

  1. sudo perl -MCPAN -e "install Net::SSH::Perl"

Now you can decompress the script and configure it

  1. tar -xzpf ssh_batch-0.1.tar.gz
  2. cd ssh_batch-0.1

put the commands in commands.txt ( one command per line )
put the servers in servers.txt (one per line ) in this format: user@hostname:port,password

Now you can test the script: run ./ssh_batch.pl and look at the output in log.txt

Warning! You can destroy multiple servers with this script!

Yeah it can do that if you're not carefull about what commands you tell it to run and you log in with a user with too  much permissions. So make sure you know what you are doing before you run it on production servers.

Atomic SCP and SFTP upload

Problem

You transfer files over scp or sftp to a server and there you have a script that processes the new files. you want the script to only start processing the files once they are completely transferred.

There's no way of knowing when the files have been fully transferred, both sftp and scp would create the files as soon as the transfer begins and will close then when it finishes. So between the time it creates them and until it closes them the files are incomplete.

Solutions

There's an easy solution: upload a lock file before you start uploading the real files and remove the lock file after the upload is finished. Modify your processing program/script to look for a lock file and only start processing if the lock file does not exist. This is good if you can modify the upload and processing scripts/programs but that's not always the case.

The harder solution involves modification to openssh source code. I created a patch that modifies scp and the sftp server so that for every file received the server will actually put the contents in a temporary file and only move the file in the real destination when/if the upload is complete.  The move operation ( rename ) is atomic only when moving the files in the same filesystem but that's not  a big problem cause we can configure the tmp location to be on the same filesystem.

Both scp and sftp server were modified so you get similar functionality by using any of them.

Installation

This patch was tested with openssh 4.6.p1. It may work with newer versions but first you should try with the same version so download the source code for 4.6p1 and decompress it.

Download my patch: SCP/SFTP atomic patch-4.6p1 (5.95 KB)

Apply the patch:

  1. cd openssh-4.6p1
  2. patch -p1 < openssh_scp_sftp_atomic.diff

Then run configure with whatever parameters you want, make and install it.

Configuration

By default scp and sftp-server will use /tmp as the temporary location where they save files till the upload is complete.

If /tmp is not on the same filesystem and the actual file destination then you have to specify a different temporary location in order to make this really atomic.

For sftp-server you can do it by adding another parameter to the Subsystem line in sshd_config

It normally looks like this (on gentoo x86_64) :

Subsystem sftp /usr/lib64/misc/sftp-server

or ( on ubuntu 9.04 )

Subsystem sftp /usr/lib/openssh/sftp-server

You have to add " -t /new/tmp/location " to that line ( without the quotes )

/new/tmp/location should be on the same filesystem as the real destination.

For example if you have /home mounted on a separate partition and you upload in /home/user  you should create a temporary folder in /home and set that as the folder to be used by sftp-server.

  1. mkdir /home/tmp
  2. chmod 1777 /home/tmp # all write/read and sticky

And the configuration line should be something like :

Subsystem sftp /usr/lib/openssh/sftp-server -t /home/tmp

Scp also needs special configuration if you want to set a different temporary location but in this case we could not just pass a special parameter to it because the scp client would not allow that so I had to make a wrapper for the scp program on the server.

The wrapper would just pass the custom temporary location in a environment variable then call the actual ( patched ) scp program.

I had scp in /usr/bin/scp so I moved that in /usr/bin/scp.bin

and I created a script named /usr/bin/scp with the following content:

  1. #!/bin/sh
  2. export TMP=/home/tmp
  3. scp.bin $@

all that's left to do is:

  1. chmod 755 /usr/bin/scp

That's it! Now you have atomic uploads for scp and sftp.

This week on twitter 2009-03-08

  • @Rocky1138 I get 50 spam comments/day on average, I don't have time to login and check all of them each day so akismet is really useful in reply to Rocky1138 #
  • Google reader just doesn't understand when I say "mark all as read". Next time I log in it will still show me some items as unread. #
  • what other good feed readers are you using ? #
  • RT: @johnreese The Future Of Email Marketing Is The QUADRUPLE OPT-IN: http://quadrupleoptin.com/ [no, that's the DEATH of it] #
  • My DBI is 91.90 Find yours and manage your followers at http://tweetsum.com #DBI #
  • @nonsequitir I'm looking for something browser based in reply to nonsequitir #
  • @adriana_iordan Thanks, I thought about trying bloglines but I might take a look at the others too in reply to adriana_iordan #
  • ok now what is this DBI that tweetsum just couldn't tell me on their page ? #
  • @nonsequitir nah, I don't really like AIR and it seems t has a problem with Ubuntu 9.04 at the moment :)in reply to nonsequitir #
  • How-to: Move from Google Reader to Bloglines | Sephys Platzish http://ff.im/1hBMb #
  • @mihaibrehar am incercat bloglines. Nu-mi place ca trebui sa dau click pe fiecare feed ca sa-l citesc sau poate nu stiu eu sa-l folosesc in reply to mihaibrehar #
  • @nonsequitir tried bloglines so far unsatisfied...will continue :)in reply to nonsequitir #
  • @smmehadi Thanks. I'll do that when/if I decide to try it again in reply to smmehadi #
  • @problogger noone uses linkedin. It's just for showing off your "connections" in reply to problogger #
  • @problogger re "who's using linkedin" - http://tinyurl.com/cpms8t #
  • @Rocky1138 Really popular blogs can get hundreds to thousands of spam / day. Try to find legitimate comments in that mess in reply to Rocky1138 #
  • if you're a graphic designer don't put all of your work in your portfolio, show only the best! #
  • @robbarrett add a parrot somewhere :)in reply to robbarrett #
  • @denisecox I totally agree. Not one real person in 10 years sent me a direct/personal email with my name in the subject line in reply to denisecox #
  • if you're objecting if to "work in a multinational company" why would you apply to work in a 2 man company? #
  • @denisecox increased open rates ? in reply to denisecox #
  • @robbarrett yeah a parrot that hands him some kind of dentist tool maybe those mirrors they use to look at the teeth :)in reply to robbarrett #
  • @mihaibrehar merci de notificare despre link. As fi vrut sa vin la geekmeet dar in din 21 pana prin 26 voi fi plecat in reply to mihaibrehar #
  • @denisecox wow a lot of your messages have your name in subject! And yes I can see how the one with the co. name is seems more relevant in reply to denisecox #
  • is this the best business model for twitter and apps : "featured/recommended users" ? seems like a lot of them are doing it #
  • @Jesse how do you pass only some of your likes to twitter ? is there some setting on friendfeed or just manually posting them to twitter? in reply to Jesse #
  • @Jesse this kind of limits my linking habits. I use likes for things I might want to look at later in reply to Jesse #
  • @cakemail_ceo tinymce or FCKeditor in reply to cakemail_ceo #
  • Bugmenot.com - login with these free web passwords to bypass compulsory registration [pic] http://ff.im/1k1wL #
  • @cakemail_ceo sorry ...wrong advice ... I thought you were looking for something else in reply to cakemail_ceo #
  • @cakemail_ceo you should look into Cforms .It's a wordpres plugin but I bet it can be adapted for other stuff in reply to cakemail_ceo #
  • decided to give AIR + twhirl another try ... upgraded twhirl and it just worked http://ff.im/1k2Cq #
  • some blogs just don't want my comments. That's why they require registration for comments #
  • MattKruse.com & Blog Archive & PHP Excel Reader [pic] http://ff.im/1k5Bf #
  • @RobOwen exactly. Plus it's really not justified. Akismet and other tools are doing a pretty good job at killing spam. in reply to RobOwen #
  • viddler users are so spammy. Made an account a few days ago and I already got 2 friend requests. I have nothing in my account #
  • RT: @problogger: Cool Tool: Twimailer - http://twimailer.com/ - more info in the emails you get when someone follows you #
  • ok twitmailer is cool but has anyone realized that you could lose your account with this ? #
  • the only thing that tells twitter that you are who you say you are is your email address #
  • you replace your email, with someone else email address and they can easily change the password #
  • @problogger what do you think about the security implications of using twimailer? IMO it seems worst then someone finding your password #
  • Configuration mangement concepts for database objects http://ff.im/1lhhP #
  • I just love XPath #
  • @spam @TwitRel #
  • anyone knows how to use functions like substring with php's DOMXPath ? #
  • it seems like substring and others are Xpath 2.0 specs and DOMXPath only supports Xpath 1.0 . What a shame? #
  • here's how @jamesdickey starts his day: "Good morning, everyone! How may I help you today? Need a RT,..." that's how you get 15k followers:) #
  • I wish I could tell stumbleupon to search only within the items I liked #
  • do you care about the economy? here's who you should follow: @economy_ms #

Powered by Twitter Tools.

Adding new php syntax

This is a quick patch I did to php's source code to implement some special syntax. Basically I wanted to be able to define an array like this:

  1. $a=[ 1,2,3,4];

Get the patch here:  New array syntax for php-5.2.6-5.2.9 (997 bytes)
To get this behavior download php's source code, extract it put my patch in the source directory and  do this:

  1. patch -p1 < new_array_zend_syntax.patch
  2. cd Zend
  3. ./buildconf
  4. ./configure
  5. make
  6. cd ..
  7. ./configure
  8. make
  9.  

( This will only compile but not install - in case you want that just type make install at the end )
And here's a test script :

  1. $a=[ 'key' => 'value',1,2,3];
  2. print_r($a);

Save this script as test.php then run sapi/cli/php test.php

This was tested on php 5.2.6 and 5.2.9. I think this is probably useless for most people but it was fun to write so why not share it :)

Review: The Hacker’s underground Handbook

Last week I was contacted by David Melnichuk to offer me a partnership/affiliate account for his ebook "The Hacker's underground Handbook". I said to David that I would prefer to see what's it all about before I would promote anything, but I would be willing to write a review and maybe promote it if he sends me a copy. He did that so here is my review ...

The pitch

I recently created the eBook: The Hacker’s Underground Handbook. It is targeted more towards people that are new to the hacking/security scene and still don’t know where to start. Although it is mainly targeted towards newbies, it also has content that will be valuable for intermediate skill levels. The product is completely legal. Once the product is purchased, the user will be taught to not abuse the knowledge gained, and the penalties if he/she chooses too.

After looking over the book briefly I completely agree that this is an ebook for newbies.

Also the phrase "learn what it takes to crack even the most secure systems" from the cover, over estimates the content. You would most likely not be able to crack the most secure systems only with the information in this book, but the author advices the reader to learn more and not rely only on the information in the book and even provides some links to more resources.

Content

I think the term hacker applies more to someone like Richard Stallman, Alan Cox, Linus Torvalds, etc then to someone like Kevin Mitnik. So in my opinion the book is more about cracking then hacking, but most people(newbies) don't know the difference so I'm not going to insist on this.

The book covers topics from  installing a linux distribution (with screenshots  a la howtoforge ) and password cracking to packet sniffing, using exploits, web site cracking, wifi cracking and social engineering ( which IMO is not really cracking but just a nerdier/l33t word for "lying" )

I like the fact that the book also offers some advices / countermeasures even if in some cases it doesn't present the most secure or all options.

Conclusions

If you're into cracking and you already know how to do a lot of stuff I would not recommend the book as there are no advanced techniques in the book but if you're new to this or you would just like to know how some things are done and how you can prevent some security incidents then the book offers a good collection of common cracking techniques.

The ebook comes with a bonus ebook named "1000 Hacking Tutorials Leaked", so for the price of $18.89 is probably a very good deal.

Click here to get the ebook ( yeah that's my affiliate link )

Have you read this book? I'd love to read your impressions about it in the comments.

This week on twitter 2009-03-01

  • RT: @AaronMartirano: People often say that motivation doesn't last. Well, neither does bathing -- that's why we recommend it daily. #
  • @techxav for a country of 47 mil it's really not so much but yeah some people think Africa doesn't even have internet :)in reply to techxav #
  • @BradHoward it's very annoying when you open up a web page and the video just starts playing. in reply to BradHoward #
  • tried the friendfeed comments wp plugin but for some reason it doesn't find any likes/comments on my blog posts... http://ff.im/1bDg7 #
  • @ludwikc why is that? is it because of a bigger challenge in running a non profit? in reply to ludwikc #
  • anyone wants a chi.mp invite? dm me withyour email address. #
  • @rajupp sure thing. check your email in reply to rajupp #
  • ok, got 3 more invites for chi.mp :)#
  • @meannie will need your email address for that. You can dm me if you don't want everyone to see it. Don't worry I don't spam :)#
  • @meannie aii sorry I forgot you can only dm if I follow ... in reply to meannie #
  • RT: @unmarketing: Daily reminder to take 5 mins and read other tweets and comment/reply/retweet. Nothing about you. Build, interact, give. #
  • if wp-super-cache is locking your blog, change the $sem_id value in wp-cache-config.php #
  • @CommentLuv friendorfollow.com in reply to CommentLuv #
  • can anyone explain how come US has a military base in Guantanamo ? I thought US as Cuba were kind of enemies #
  • is anyone still using WAP ? Isn't it obsolete now that we have the new generation of smart phones? #
  • RT: @maydbs: "Life is like riding a bicycle. To keep your balance you must keep moving." Albert Einstein #quote - Waking up... =] #
  • RT: @Themelis_Cuiper: Planecrash on Schiphol Amsterdam Airport , Turkish airlines plane has crashed, airport and roads closed for emergency #
  • thanks to dnsstuff for having a useless Free Health check #
  • intoDNS: checks DNS and mail servers health http://ff.im/1daq4 #
  • @danielvoicu dude that's so cool. dar raspunde destul de greu la comenzi in reply to danielvoicu #
  • How to Use Twitter Without Twitter Owning You - 5 Tips http://ff.im/1dw17 #
  • Reinventing the Kindle (part II) http://ff.im/1dvVn #
  • @Themelis_Cuiper I consider #spam these types of messages in reply to Themelis_Cuiper #
  • I just realized: twitter is just newsgroups 2.0 #
  • well not really just twitter but twitter + friendfeed + all other status update services #
  • now all we need is the actual groups and some file sharing p2p system based on it :)#
  • @thetylerhayes the newsgroups were / are communities just like the ones you have on twitter but without the "social networking" part in reply to thetylerhayes #
  • @thetylerhayes yeah but newsgroups were more portable, just like twitter because of the api you can post from anywhere in reply to thetylerhayes #
  • @thetylerhayes also like twitter newsgroups didn't have a business model ... actually they did but a not very good one anyway :)in reply to thetylerhayes #
  • RT: @unmarketing: RT @loubortone: #definetwitter Not "what are you doing?" should be "why aren't you working?" #
  • ok, I'm really sick of this ... from now on any dm I get that I feel like it's automated will trigger an unfollow #
  • @LorenaHeletea ma bucur Lorena. Ce lucruri ai invatat mai exact? in reply to LorenaHeletea #
  • why do people have to participate in these #followfriday things it's so annoying to have your stream full of these tweets. I never click one #
  • The panhandler's secret http://ff.im/1fyJt #
  • @bwhalley I'm not really against following people but having my stream full of ids isn't going to help me decide who to follow in reply to bwhalley #
  • @jaydeflix true but I was looking not only at twitter but also the other networks and the data sharing that goes between them in reply to jaydeflix #
  • I'm looking to buy professionally designed email templates. DM with offers. Please retweet! #
  • @tamaragielen thanks in reply to tamaragielen #
  • @DominiqueGoh I would rather have in in the message. If you just drop a link and very little text it's more likely to be considered spam in reply to DominiqueGoh #
  • @tamaragielen seems like the templates on templatezone only work with their HighImpact Email product. in reply to tamaragielen #
  • @DominiqueGoh I actually unsubscribe from those newsletters that only put links to online articles in them. in reply to DominiqueGoh #
  • @DominiqueGoh try nvu in reply to DominiqueGoh #
  • Captivating Romania & Blog Archive & The Merry Cemetery, SăpânÈ›a http://ff.im/1gjeV #
  • RT: @ludwikc: "Removing security on Amazon purchased e-books" - oldschool *.txt report - http://bit.ly/ZBxX9 #
  • RT: @improvingtheweb: 6 jquery chart plugins: http://tr.im/gS6C #
  • @AlexVolocaru sure, but not evey customer wants to spend tons of money on email design in reply to AlexVolocaru #
  • @AlexVolocaru in most cases replacing some images and colors in a template works just as well as doing it from scratch in reply to AlexVolocaru #
  • firebug is a life saver!!! #
  • @DaivRawks wow her website actually improved since the last time I visited :)in reply to DaivRawks #
  • RT: @kevinsenne: Funny and very true post. 50 Signs that You're a Spammer. We're right there with you Joe. http://bit.ly/rcf6u #
  • @AlexVolocaru I agree about the usability problem but why can't templates provide a good experience? I mean really professional templates in reply to AlexVolocaru #
  • @AlexVolocaru as for the relevance I think the content is the one that counts here in reply to AlexVolocaru #
  • @kevinsenne deliverability=messages in inbox/total messages delivered . Returned messages ( bounces ) don't count as delivered in reply to kevinsenne #
  • @CraigTeich you following me back? :)in reply to CraigTeich #
  • @AlexVolocaru I'm talking about small businesses that can't afford to hire a branding firm in reply to AlexVolocaru #
  • @kevinsenne I agree, that's why I said it should not count. But measuring this way deliverability exactly is practically impossible. in reply to kevinsenne #
  • @AlexVolocaru @kevinsenne the open rate is more important, but it's not always good metric for ESP performance in reply to AlexVolocaru #
  • Personal branding in the age of Google http://ff.im/1gBDb #
  • should you do business with totally clueless clients ? #
  • "We need someone to clean up our deliverability. Ensure we are accepted by the search engines for deliverability" What ? #
  • how much do you trust akismet? do you watch your spam comments for false positives? #

Powered by Twitter Tools.