Kevin Minnick's Blog

How to disable root SSH access to your Linux servers

In my previous post I wrote about how to slow down brute force SSH attacks.  If you'd like to add a bit more security, and do what my Intro to CS college professor told me was "good practice", don't use root to log in to your server.  These steps will show you how to disable root ssh access, but still have the ability to perform administrator level tasks.

WARNING: The following steps have the potential to lock you out of your Linux server if you're not careful.  I suggest practicing on a new virtual server first so that you can easily re-kick the machine if necessary.  I am using Ubuntu, your Linux flavor may have slightly different instructions.

Let's take a look at the steps from a high-level, and then deep-dive into the details:

1. Add a new user account that will be used to log in
2. Add the new user as a 'sudoer' (someone who can perform administrative tasks)
3. Test to make sure you can log in as that user, and can perform admin tasks (sudo)
4. Invalidate the 'root' user password
5. Test to make sure you are not able to log in as root

Create a new user, fred

$> useradd -d /home/fred -m fred $> passwd fred //sets the password $> vi /etc/passwd //fred prefers bash fred:x:1000:1000::/home/fred:/bin/bash

Add Fred to the sudo (admin) group

$> vi /etc/group sudo:x:27:fred

Verify the 'sudoers' file is correct

$> visudo %sudo ALL=(ALL:ALL) ALL

Log out as root, log in as Fred, verify sudo works

$> exit $> ssh fred@yourserver $> sudo vi /etc/passwd

Invalidate the root user password

$> sudo passwd -l root Logging in as 'root' is now disabled via SSH, unless you have set up SSH keys in the past. If you have, you'll need to remove them to fully remove access.

How to block brute force ssh attacks with fail2ban

I'm sending the log files from my Rackspace Cloud Servers to loggly, as I described in a previous post. Take a look at this graph from the loggly dashboard:

 

Uh oh, something looks suspicious here.  Clicking on that spike right after midnight shows the problem:

Someone is trying to break into my server by attacking ssh with many login attempts in a short period of time. Looks like I need to install fail2ban. fail2ban will keep an eye on my log files and set up iptables rules after a configurable number of failed attempts. It will also remove the rules after a specified period of time. apt-get update apt-get install fail2ban Configure /etc/fail2ban/jail.local appropriately. On Ubuntu 11.04, this was changing the [ssh] logpath = /var/log/auth.log to [ssh] logpath = /var/log/messages And restart: service fail2ban restart I test it from a different server and it works:

 

From the original server you can see fail2ban added the iptables rule:

And after 5 minutes, I'm allowed back in:

I should see a big drop in login attempts over the next few days.

Book: Zen and the Art of Motorcycle Maintenance

My first book for 2012 turned out to be a fun but challenging read. A novel unlike any I've read before; mixing the topics of motorcycles, values, fatherhood, philosophy, and craftmenship all into one.  

The book was originally published in 1974 and is considered a modern marvel.  I especially liked this book because the topics of craftmenship, quality, and art can easily be applied to the field of computer science.   This book will certainly stretch your mind but I will warn you that it's a challenge to read because of the deep dive into philosophy.

My Favorite Books from 2011

Occassionally I get asked for book recommendations.  Here's a somewhat comprehensive list of the books I read in 2011.  I don't think there are any I would not recommend reading, but I've tried to stack-rank them according to how much I enjoyed reading them.

Non-Fiction

• Death by Meeting
• Crucial Conversations
• The Four Obessions of an Extroadinary Executive
• Silos, Politics and Turf Wars
• The Five Temptations of a CEO
• Rich Dad Poor Dad
• Getting to Yes
• Do More Faster
• Delivering Happiness
• The 4-Hour Workweek
• Inspired: How to Create Products Customers Love
• Switch: How to Change When Change Is Hard
• Doing Both
• The PayPal Wars

Coding Related:

• Cocoa Programming for Mac OS X (4th Edition)
• Objective-C Programming: The Big Nerd Ranch Guide
• Advanced Mac OS X Programming: The Big Nerd Ranch Guide
• Programming in Objective-C (3rd Edition)
• Cocoa and Objective-C: Up and Running
• Hadoop: The Definitive Guide
• Apple Training Series: AppleScript 1-2-3

 

Fiction

• Matterhorn
• Unbroken: A World War II Story of Survival, Resilience, and Redemption
• The Girl with the Dragon Tattoo
• The Girl Who Kicked the Hornet's Nest
• The Girl Who Played with Fire
• Water for Elephants
• The Help
• Hunger Games
• True Grit
• Cross Fire (Alex Cross)
• One Summer
• The Dark Tide
• Indulgence in Death
• Jonathan Livingston Seagull

 

Add Beautifully Highlighted Code to Your Website with SyntaxHighlighter

I wanted to find a better way of formatting the small code snippets that I post to this blog to make them easier to read and understand. SyntaxHighlighter seems to be the answer and here are the steps I used to add it to my typepad blog.

Example w/ PHP

<code class="brush: php">
//code snippet to list the files in a container
//on Rackspace Cloud Files
//if container doesn't exist create it, and make it public
try {
    $container = $conn->get_container('my_container');
    $obj_list = $container->list_objects();
    print_r($obj_list);
}
catch (Exception $e) {
    $container = $conn->create_container('my_container');
    //make container public
$container->make_public();
}
</code>

Displays as:

//code snippet to list the files in a container //on Rackspace Cloud Files //if container doesn't exist create it, and make it public try {     $container = $conn->get_container('my_container');     $obj_list = $container->list_objects();     print_r($obj_list); } catch (Exception $e) {     $container = $conn->create_container('my_container');     //make container public $container->make_public(); }

Installation steps for TypePad:

1. Download the code
2. Follow the build instructions in build/README.txt
3. Upload the following files to your File Manager in TypePad (see screen shot) from the build/output directory
4. Add a new Custom HTML Module to your blog at the very bottom (see code below)
5. Create a new post on your blog with the above code snippet

List of files needed

Code for Custom HTML Module

<!-- Include required JS files --> <script type="text/javascript" src="http://<yourwebsitehere>/sh/shCore.js"></script> <!-- Include Brushes --> <script type="text/javascript" src="http://<yourwebsitehere>/sh/shBrushPhp.js"></script> <script type="text/javascript" src="http://<yourwebsitehere>/sh/shBrushBash.js"></script> <script type="text/javascript" src="http://<yourwebsitehere>/sh/shBrushAppleScript.js"></script> <script type="text/javascript" src="http://<yourwebsitehere>/sh/shBrushCpp.js"></script> <script type="text/javascript" src="http://<yourwebsitehere>/sh/shBrushDiff.js"></script> <script type="text/javascript" src="http://<yourwebsitehere>/sh/shBrushJava.js"></script> <script type="text/javascript" src="http://<yourwebsitehere>/sh/shBrushJScript.js"></script> <script type="text/javascript" src="http://<yourwebsitehere>/sh/shBrushPerl.js"></script> <script type="text/javascript" src="http://<yourwebsitehere>/sh/shBrushPlain.js"></script> <script type="text/javascript" src="http://<yourwebsitehere>/sh/shBrushPython.js"></script> <script type="text/javascript" src="http://<yourwebsitehere>/sh/shBrushSql.js"></script> <script type="text/javascript" src="http://<yourwebsitehere>/sh/shBrushXml.js"></script> <!-- Include default theme --> <link href="http://<yourwebsitehere>/sh/shCore.css" rel="stylesheet" type="text/css" /> <link href="http://<yourwebsitehere>/sh/shThemeDefault.css" rel="stylesheet" type="text/css" /> <!-- Options --> <script type="text/javascript"> SyntaxHighlighter.config.tagName = "code"; SyntaxHighlighter.config.stripBrs = true; SyntaxHighlighter.all(); </script>

DIY: How to upgrade the memory in your iMac AND save $350

My iMac came with 4GB of RAM when I originally purchased it in late 2009.  That amount has been more than enough until recently. Now that the entire family is more than proficient using the iMac, there never seems to be enough RAM available, bringing the machine to a slow crawl.  The main culprits are usually Chrome, Firefox, and photo editing software such as Aperture or Photoshop.  Having four people all logged into the machine at the same time uses a lot of memory.  I decided that it was time to end the frustration and upgrade the machine to 12GB of RAM.

There's really 2 ways I could accomplish this task:

1. Use the local Apple store.  This option didn't appeal to me because of the hassle of lugging the 21" monster to the mall and back.

2. Buy the memory online and install it myself.  I do have a degree in Computer Science, how hard can it be?

So I started researching option #2.  After some quick searching I found the memory needed for my iMac model on the apple.com website for $400 (I needed 8GB total).   Woah, really?  That seemed rather high.  I then found the crucial.com website and found the same memory there for $45.  Wow, what a huge difference.  The crucial.com website also had a really awesome system scanner that pulled up the memory chip that was needed for my late 2009 iMac.  Then it quickly gave me a few options and the cost for each option.  I was really impressed with the website and the experience.

I placed my order online and within a week my memory arrived via UPS.  I then pulled down the instructions on how to install the memory from the Apple Knowledge Base.  The instructions were really well done and easy to follow.  Here's a link to the version for my model.

Overall, there's a lot to learn from both Apple and Crucial.com about user experience and design.  Adding memory to your computer can be a very intimidating and confusing task.  Apple had enough foresight to put the location of the memory chips in an easy to access location.  Unlike the typical PC, I did not have to take the entire computer apart to install more memory.  I simply had to remove three screws from a bottom panel and insert the new chips.  Very easy and great instructions with diagrams in the knowledge base.

Crucial.com knows that it's very difficult to find the right memory chips for your computer model.  They created a great tool that determines the right chip for you and quickly give you easy to understand options in a nice layout.  They removed many of the barriers and confusion from the purchasing process.

Screen shot of memory usage before upgrade:

Screen shot of memory usage after upgrade:

 

Notice the 0 bytes of Swap used, nice.

Crucial.com website:

Photos of install process:

 

 

Startup Advice: Focus On Your Core Values Early

Last month I had the privilege to attend a 2-day Core Values offsite with 40 of my fellow Rackers (Rackers are what we call Rackspace employees).  Rackspace has about 4000 employees so being selected as one of the 1% to help think about our Core Values was quite an honor. 

The session was NOT about defining or debating our Core Values; that was done over 10 years ago in the early days of Rackspace.  This session was focused on how we can continue to stay committed to our Core Values as we grow.  It was a fantastic two days of spirited discussion and I was amazed at the amount of passion and talent in the room.

I would encourage every startup to spend time thinking hard about their core values as early as possible.  In bold below I highlight 12 tips for defining and living your Core Values.

Your company’s Core Values should reflect who you really are as a company.  This is critical but hard to define if you are just getting started.  I don’t expect many founders to be able to sit down day 1 and define their Core Values.   But certainly within the first two years you should be able to define whom you are and what kind of company you are trying to build.  If your Core Values don’t reflect who you really are and the way your company operates they are meaningless.

As you are defining your Core Values, you need to make them unique and memorable.  Copying some other company’s Core Values isn’t going to work.  And if your Core Values are too long and wordy to remember they are of no use.  Simple, memorable phrases work best.  Zappos has an excellent example of this: http://about.zappos.com/our-unique-culture/zappos-core-values.

Although Core Values should be short and memorable, it’s important to have additional material available that defines them more thoroughly.  I suggest building a “What it is / What it is NOT” for each Core Value.   Take a somewhat complex Core Value like “Full Disclosure and Transparency”.  You should take the time to write out 5 example behaviors that it is, and 5 example behaviors that it is NOT.  I’ll give you one each.  It is “having integrity, being direct and honest.”  It is NOT “being quiet in a crisis or when things go wrong.”  This was a key part of the offsite this month and I believe it’s critical to get your employees involved in this process.  They will have more context, buy-in, and understanding if they can participate in defining these behaviors.

Your Core Values should be re-enforced constantly.  Posters and t-shirts help, but that’s not enough.  On an employee’s first day they should go through Core Values training.  There should be a mechanism available to call-out Core Values violations and hold each other accountable.  On the flip side, there should be a way to celebrate when things go well, and point out people living the Core Values.

Constant re-enforcement of your Core Values will help your employees in everyday situations.  Interviewing based on your Core Values should be an important part of your interview process.  Your employees can protect your culture as you grow using your Core Values.  At the same time though, Core Values should never be used as weapons.  This often occurs when employees need to have something go their way in a meeting and use a Core Value inappropriately as an argument.  Be on guard for this type of behavior as it can quickly damage your culture.

Your Core Values should be sacred.  Think of them like Constitutional Amendments, it should take an act of Congress to change them.  This is why it’s so important to spend a lot of time on getting them right early and making sure they reflect who you really are.

My final piece of advice is that over time, stories really matter.  Take the time to document your best stories of employees living your Core Values.  These stories can be passed from generation to generation.  This will help protect your company’s culture as you grow.  My favorite stories at Rackspace are about Rackers helping other Rackers out in times of personal need, truly living up to our “treat Rackers like Friends and Family” Core Value.

Pictures stolen from vminnick.smugmug.com

 

An Updated List of My Favorite Desktop Applications

Two years ago I wrote about my favorite desktop applications (was using Windows at the time).   Two years have passed and I'm now using a Mac full-time.  The death of the desktop application has been extremely exaggerated over the years.  It's been well over 5 years now since web-based apps became first-class citizens and I still prefer the desktop application over the web app in most cases.

Here is the updated list:

1. Chrome - The fastest browser out there.
2. Microsoft Outlook - When hooked into Exchange, it can't be beat for shared calendaring.
3. Microsoft Office - Really good, even the Mac versions.
4. Evernote - Sync your notes across all devices, backed up to the cloud, plus amazing search.
5. XCode - Objective-C is my girlfriend.
6. Eclipse - Great for python.
7. iTunes - I really don't like iTunes, but there's no alternative.
8. Jungle Disk - Solid back up and restore application.
9. OmniGraffle - Great for creating amazing looking graphics.
10. iStats Menu - Monitor system performance from your menu bar.

As you can see the list is quite different.  Firefox lost to Chrome.  OpenOffice lost to Microsoft Office.  I don't spend much time keeping up with Twitter and thus no tweetdeck (I also rarely use Facebook).  I do use Skype, but it's rare.  I haven't thought about Meebo in a long time.  And I switched from Blackberry to iPhone and no longer use my phone as a tether to the Internet, WiFi is usually available everywhere these days.

So, what's out there that I'm missing out on? 

 

Picture borrowed from vminnick.smugmug.com

 

I’ll admit it when I’m wrong, why I love my Mac

On November 27, 2009 I wrote about why I chose a Windows 7 PC over a Mac.  Ooops, that was a mistake.  Late in 2010 I got a Mac and I’m so glad I made that decision.  It’s not that Windows 7 is bad, it’s that the Mac is so much better.  As for that Sony, I gave it to my sister, who loved it.  A burglar broke into her house though shortly after I gave it to her and stole it.  I hope it eventually found a good home.

I break the advantages of a Mac down into 3 categories: Little Things, Unix, and Hardware.

Little Things

It’s the little things that make such a big difference in the experience.  Here are some examples of little things I so love:

• When installing an application you aren’t bombarded with endless pop-up messages about “are you sure, this could be the end of your computer”?
• I can close my laptop at work, take it home, open it back up and pick up from where I started instantly.   Crazy fast.  I don’t have to think about Standby versus Hibernate versus Sleep versus Shutdown versus low energy modes.
• Time Machine – A fantastic backup solution (and network router / printer sharing device).
• Keyboard Shortcuts – The function keys at the top of the Mac keyboard are extremely useful.  I never used these on a PC but on my Mac I use them all the time.  Screen brightness, volume, and iTunes play buttons are very handy.
• Spotlight search really does work.  I can find what I need really quickly.
• Multiple screens (spaces) are a snap. (Command+Left/Right/Up/Down)
• Xcode – I was first intimidated a little by Xcode, but now that I’ve been using it for several months it’s by far my favorite IDE of all time.  It’s really simple to create Mac applications using Xcode.  Objective-C is my new girlfriend.
• Fast window switching – at first I couldn’t get how window switching on the Mac worked, but then I learned how to do Command+~ and now I’m a big fan.
• Malware / Spyware – I’m still shocked how many people on Windows open attachments they shouldn’t.  While I know this can and does happen on Mac’s, I hear about rarely.  I’m not sure if Mac users are smarter or there are just fewer attempts by the bad guys to hit Mac users.

UNIX

I’m going to geek-out a bit here (as if I haven’t already), but I’m not sure how I ever survived years of Windows and not having a native Unix shell.  It’s amazing how powerful and convenient it is to be able to use the same commands on my desktop as I do on my servers.  I’m a huge Python fan and having the ability to script little things quickly is really powerful.  Built-in ssh, where have you been all my life? 

Hardware

Let’s start with the screen on my Mac Book Pro.  It’s gorgeous.  I work on this screen all day and it’s really crisp and sharp.  I look at some of the laptop screens I’ve used in the past and I’m amazed I still have 20/20 vision. 

The keyboard types great and all of the keys are just where I expect them.  Nothing use to frustrate me more than PC laptops that saved space by moving keys like Shift and Function to weird places.

I’m use to hard drives in laptops that I can literally feel their pain.  I can hear them churning and cranking and spinning and working so hard to get the info that Windows needs.  Not on this machine.  I never think about the hard drive.  Never hear it, never wonder what it’s doing that is so hard.  Usually after a year or so on a PC you have to defrag or re-install to get back to the speed when you first bought the machine.  I’ve been using this Mac for over a year and it’s just as fast today as the day I got it.

The battery life is much better than I’m use to on Sony or Dell.  I can go a whole day at work of just carrying my laptop around.  I can code for hours in a closet at home hiding from my kids without being plugged in.

Another little thing is the power supply.  More specifically, the little piece that plugs into the laptop.  It’s magnetic!  It detaches so easily and that’s important for a number of reasons, including the ability to yank the laptop away from your 6 year old quickly.  It also is compatible with the Mac-Book Air power supply.   You can flip that little piece upside-down if needed, depending on where your electrical outlet is located.  Very nice!  It has a little green light when it’s working.  I’ve actually found that useful on more than one occasion.

Finally, the built-in microphone has amazing quality.  I make tons of calls from my laptop and callers cannot tell that I’m using a laptop to make the phone call.

On Microsoft

I’m still a big Microsoft fan.  I think Microsoft makes great products.  Microsoft Office, Exchange, SQL Server, Active Directory, and .NET are some of the greatest pieces of software ever.  Ever.   I would even throw Windows into that category.

But Microsoft doesn’t have the total package.  They don’t make the hardware, they don’t support Unix, and they don’t have an ecosystem.  The magic is when you combine the hardware with the software with the ecosystem (iPods, iPhones, iPads, Apple Tv, Time Machine, iCloud, etc).

 

Random Linux Commands I Used Today

Here are a few random Linux commands I needed today that you might find useful...

I had a mysql script that only supported unix sockets. But the mysql server I needed to connect to was not local and I did not have local access to the server.  So I needed a way to "fake" a local socket that actually connected to a remote server via tcp.  Enter "socat":

/usr/bin/socat -d -lf/var/log/socat.log UNIX-LISTEN:/var/run/mysqld.sock,reuseaddr,fork,unlink-early,mode=777,su=nobody,user=mysql,group=mysql TCP4:50.123.205.190:3306 &

Next, I needed a quick way to block ICMP or ping traffic to my server.  Here's an iptables rule that did the trick:

iptables -A INPUT -p icmp --icmp-type echo-request -j DROP

Keep your Ubuntu system updated with the latest patches:

apt-get update ; apt-get upgrade

I needed to get the mac address for a server:

ifconfig -a

I wanted to stress-test a new mysql a bit:

mysqlslap -h 50.123.205.190 -u root -p --auto-generate-sql -vv --concurrency=300 --engine=innodb

but this test wasn't quite doing it for me, so I found a better benchmark tool:

sysbench --test=oltp --mysql-table-engine=myisam --oltp-table-size=1000000 --mysql-host=10.18.64.189 --mysql-user=mysql --mysql-password=xxx prepare

sysbench --test=oltp --mysql-host=10.18.252.22 --mysql-user=mysql --mysql-password=xxx --oltp-read-only --num-threads=200 --max-requests=10000 run

I wanted to run a quick scan on which ports were open on my server:

nmap -v 50.57.94.171

An more "graphical" version of top called htop:

And finally, for those who haven't tried out iptraf to see your network traffic, take a look: