Command Line on MarkJacobsen.net

Use SCP to copy files between remote and local hosts

Tue, 05 Feb 2019 01:44:54 +0000

So you need to copy a file you have on a remote server locally, or send a local file to a remote server? Fortunatally, it’s pretty easy with scp. Here’s an example to copy a remote server to the local machine.

scp -P 65124 remoteuser@remotehost:/var/lib/path/to/file.txt /var/lib/path/to/

…which will connect to “remotehost” on port 65124 as “remoteuser” and copy the /var/lib/path/to/file.txt on “remotehost” to the local /var/lib/path/to/ directory.

Need to copy a local file to a remote host? Just switch the parameters up!

scp -P 65124 /var/lib/path/to/file.txt remoteuser@remotehost:/var/lib/path/to/

Execute MySql script from the command line

Mon, 19 Mar 2018 10:24:00 +0000

This tip is very similar to how to “[Execute a SQL file via the MySQL command line in one line][1]” only this time specifying the DB you want to execute the script against and being prompted for your password…

mysql -u user -p -h localhost MyDb < ~/SQL/script.sql[1]: http://markjacobsen.net/2012/12/execute-a-sql-file-via-the-mysql-command-line-in-one-line/

Using sed to view and replace text on a specific file in Linux

Sat, 17 Mar 2018 14:26:00 +0000

Ever need to replace very specific text on a particular line in a very large file on Linux? Enter the sed command.

To view the line (let’s say line 864 in this case) in the specific file, issue this command…

$ sed -n '864'p myfile.txt

And let’s say you get back the string ” I hate cheese”. Since we all know that’s not true, you can use a different variation of the sed command to replace “hate” with “love” like so…

$ sed -i '864s/hate/love/' myfile.txt

And if you’re really curious on how this even came about, it was due to an error in the phpMyAdmin export.php script when you had to change “break 2” to “break” on line 846 as identified here: https://askubuntu.com/questions/928883/500-error-on-phpmyadmin-export-on-phpmyadmin-export-php/930975

OwnCloud webapp not reflecting true list of files

Mon, 06 Jun 2016 18:54:00 +0000

If you’re using OwnCloud and have a process whereby your add or remove files from directories via some sort of server process, you may find that you need to force OwnCloud to update/rescan the file listing. To do so you can manually run the following command…

sudo -u username php /path/to/owncloud/console.php files:scan --all

You may also want to consider adding it to the crontab for “username” on a daily basis.

Thanks to this site for pointing me in the right direction.

Sendmail isn’t sending to domain.com : Linux

Thu, 02 Jun 2016 12:31:00 +0000

Say you’re using sendmail to relay email on a web server that you own that’s named “domain.com”, and email is being delivered fine to any domain except domain.com. What could be the problem?

Rename your server. No server should be named with an actual domain name.
Change /etc/hostname to the new name for your server
Update /etc/hosts to remove the domain name and replace with the new hostname
Remove the domain name from /etc/mail/local-host-names
Restart sendmail “sudo service sendmail restart”
Restart your server “sudo reboot”

Once you’ve done that test sendmail out like so…

echo "command line test" | mail -s "Sendmail test" me@domain.com

Improved Linux command history

Sun, 03 Apr 2016 23:14:00 +0000

With four lines setup one time you can type the beginning of a command (ex: “ls”) then press the up and down arrows to see the most recent commands containing the text you typed (ex: “ls ~/scripts”) making it much easier to find what you’re looking for. To make this happen, create a text file called .inputrc in your home folder and put the following four lines inside:

"\e[A": history-search-backward
"\e[B": history-search-forward
set show-all-if-ambiguous on
set completion-ignore-case on

Access a Console (including CYGWIN) from Notepad++

Mon, 12 Oct 2015 18:22:00 +0000

If you’re regularly in Notepad++ you may find the NPPConsole to be a huge help. With it you can access a windows console or even cygwin if you prefer (I do). To get started…

Download the NPPConsole Plugin
Extract the dll to your n++/plugins directory
Restart Notepad++ if it was already running
Use the Plugin Manager to make sure the plugin is available

If you want to just use the Windows console (DOS) you should be all set. To integrate with CYGWIN…

Choose Plugins -> NppConsole -> About…
In “Command to run” enter: C:\cygwin\bin\bash.exe ${–login -i}
In Line number pattern after file name enter: :${LINE}

Linux ls output colors

Mon, 23 Mar 2015 15:48:00 +0000

If you’ve used linux much and your terminal is set to display files and directories in color, you know how frustrating it can be to have dark blue text on a black background like so (I’ve seen it much worse too)…

[][1]

If you would like to modify the output so the colors are a bit more readable, you can add the following to your .bashrc file…

alias ls='ls --color'
LS_COLORS='di=33:fi=36:ex=31'
export LS_COLORS

which will produce an easier to read version like so (with directories yellow, files a crayon/teal, and executable files red)…

[][2]

As pointed out in this article…

The first line makes ls use the –color parameter by default, which tells ls to display files in different colours based on the setting of the LS_COLORS variable.

The second line is the tricky one, and what I have worked out so far has been by trial and error. The parameters (di, fi, etc.) refer to different Linux file types. I have worked them out as shown

di = directory
fi = file
ln = symbolic link
pi = fifo file
so = socket file
bd = block (buffered) special file
cd = character (unbuffered) special file
or = symbolic link pointing to a non-existent file (orphan)
mi = non-existent file pointed to by a symbolic link (visible when you type ls -l)
ex = file which is executable (ie. has ‘x’ set in permissions).

The *.rpm=90 parameter at the end tells ls to display any files ending in .rpm in the specified colour, in this case colour 90 (dark grey). This can be applied to any types of files (eg. you could use ‘*.png=35’ to make jpeg files appear purple.) As many or as few parameters as you like can go into the LS_COLORS variable, as long as the parameters are separated by colons.

Using trial and error (and a little bash script I wrote… my first one ever! 🙂 I worked out all the colour codes, at least my interpretation of them –

0 = default colour
1 = bold
4 = underlined
5 = flashing text
7 = reverse field
31 = red
32 = green
33 = orange
34 = blue
35 = purple
36 = cyan
37 = grey
40 = black background
41 = red background
42 = green background
43 = orange background
44 = blue background
45 = purple background
46 = cyan background
47 = grey background
90 = dark grey
91 = light red
92 = light green
93 = yellow
94 = light blue
95 = light purple
96 = turquoise
100 = dark grey background
101 = light red background
102 = light green background
103 = yellow background
104 = light blue background
105 = light purple background
106 = turquoise background[1]: https://i0.wp.com/markjacobsen.net/wp-content/uploads/2015/03/dirColorBefore.png [2]: https://i0.wp.com/markjacobsen.net/wp-content/uploads/2015/03/dirColorAfter.png

Determine Default Java JVM HeapSize

Sun, 11 Jan 2015 00:16:00 +0000

In the life of a Java developer every now and then you’ll have one of your apps run out of memory. While this is usually due to poor coding (not closing resources, or managing large files) it helps to know exactly what the defaults are for your system. I had just this need today so went out looking for an answer.

What I found that was the most helpful was [this page][1], but for quick reference here’s how to do it on Windows…

c:\>java -XX:+PrintFlagsFinal -version | findstr /i "HeapSize PermSize ThreadStackSize"
 
 uintx InitialHeapSize := 266634176 {product}
 uintx MaxHeapSize := 4267704320 {product}
 uintx PermSize = 21757952 {pd product}
 uintx MaxPermSize = 85983232 {pd product}
 intx ThreadStackSize = 0 {pd product}
java version "1.7.0_40"
Java(TM) SE Runtime Environment (build 1.7.0_40-b43)
Java HotSpot(TM) 64-Bit Server VM (build 24.0-b56, mixed mode)

Note that the values returned are in bytes so throw them into google to convert them into something useful 🙂[1]: http://www.mkyong.com/java/find-out-your-java-heap-memory-size/

Pass a Command Line Argument to an Alias

Fri, 28 Nov 2014 15:11:06 +0000

Ever want to pass in a command line argument to an alias? You would think you could just do it with $1, but actually you have to create a function and then call that function. So, for instance if you want to pass a portion of a log file name, you could set your alias like this…

alias catlog='function _catIt() { cat /var/logs/$1.log; };_catIt'

then, when you want to see the XX00D log, you just call your alias like so…

catlog XX00D

Checking and Maintaining Linux Disk Space

Mon, 10 Nov 2014 13:30:00 +0000

Ever need to find out what’s using the most space on your Linux box? There are a couple commands that will help make things easier…

To find out how much space you have use:

df -h .

Which will give you output like so…

Filesystem Size Used Avail Use% Mounted on
/dev/sda 20G 15G 4.2G 79% /

To list out the directories using the most space, use this handy command…

sudo du /usr/local | sort -n

… where /usr/local is the directory you want info for (you can also just start from root: /)

Learning VIM

Wed, 24 Sep 2014 11:49:00 +0000

So you would think that as a programmer for many moons now that I would have considerable experience in VIM and VI, but I’m hear today to admit that I am not. For years I’ve gotten by with my favorite Windows text editor notepad++, but the geek in me wanted to know why everyone loves Vim so much. To that end the 2 things I started with are…

Downloading the Windows vim editor from [Vim.org][1]

Working through the tutorial at http://www.openvim.com/tutorial.html

I can’t say I’m a convert yet, but it’s always nice to have another tool in the toolbelt. Here for your and my reference is [my quick VIM cheat sheet][2].[1]: http://www.vim.org [2]: http://visit.markjacobsen.net/vim-cheat-sheet/ “VIM Cheat Sheet”

Connect to DB2 via UNIX Command Line

Tue, 23 Sep 2014 11:26:00 +0000

To access DB2 via the command line on a UNIX/Linux box you need…

The DB2 client installed on the box
The DB you’re trying to connect to cataloged in the client

Once you know you have that, simply source your db2profile and connect…

. /path/to/db2profile
db2 connect to DBCATNAME user yourid using yourpw

… where DBCATNAME is the name the DB is cataloged as on that box, yourid is your user ID for connecting to the DB, and yourpw is your password for connecting to the DB. Note, that if the actual database resides on the same box, and you want to login as the user your logged into the UNIX box as, you can simplify the connect string and not have to include your user ID or password…

. /path/to/db2profile
db2 connect to DBNAME

Determine DB2 Client Version (aka: level) installed on UNIX box

Fri, 19 Sep 2014 15:10:00 +0000

Sometimes you need to know what version of the DB2 client is installed on a particular unix/linux box. To do so, simple source the client profile and run db2level like so…

. /path/to/db2profile
db2level

UNIX: Don’t Show Permission Denied Errors when using find Command

Wed, 10 Sep 2014 11:49:00 +0000

If you’re using the unix find command to search for files with a particular name

find -name theName
find: `./dir/thing': Permission denied
find: `./dir/thing1': Permission denied
find: `./dir/thing2': Permission denied

You might get distracted by all those “Permission denied” errors. The easy way to solve this is to redirect stderr to /dev/null like so…

find -name theName 2>/dev/null

UNIX: No such file or directory but the file exists

Fri, 29 Aug 2014 10:49:00 +0000

Ever have a script that is executing another script and get an error that looks like this?…

/path/stub.ksh[2]: /path/XX/script.ksh: not found [No such file or directory]

Then you make sure that the file does in fact exist, and that you can read it?

So why does it say there’s “No such file or directory”?

Might want to check if the file has Windows line breaks. Easiest way to do that is run the following…

cat -v /path/XX/script.ksh

… and you’ll probably see that your lines end with ^M characters. If so, you have Window’s line breaks and you might want to look into not introducing them in the first place (save in Unix format), or check out the dos2unix command.

Quick apt-get Tutorial

Thu, 10 Apr 2014 07:18:38 +0000

List installed packages…

dpkg --get-selections | grep -v deinstall

List installed packages (but filter based on name)…

dpkg --get-selections | grep -v deinstall | grep filter

Install a package…

sudo apt-get install the-package-name

Uninstall/remove a package…

sudo apt-get remove the-package-name

Update package lists/dependencies (does not actually install anything)…

sudo apt-get update

Apply updates to existing packages based on an “update” call (without removing anything)…

sudo apt-get upgrade

Apply updates to existing packages based on an “update” call and remove obsolete packages…

sudo apt-get dist-upgrade

What files are taking up the most space on my LINUX box?

Tue, 28 Jan 2014 20:42:50 +0000

Use du piped to sort for a nice list with the files using the most space at the bottom…

cd /dir/you/care/about
du -a|sort -n

How to tell if a Perl Module is Installed

Mon, 04 Nov 2013 21:55:50 +0000

Want to check if a perl module is installed on your system? There’s a oneliner for that! Here’s an example…

perl -e "use Text::CSV"

… which would of course check if Text::CSV could be loaded. If it can, nothing will be printed. If not, you’ll get an error with some possibly useful information to figuring out why not.

Change permissions on files of a specific type in linux

Thu, 17 Oct 2013 07:09:32 +0000

Need to change all *.ksh files to be executable under a directory, but not having luck with a recursive chmod? The issue is you need to combine chmod with a find and xargs like so…

find /home/user -name '*.ksh' | xargs chmod 744

The first piece lists all the files under the path that match *.ksh and passes them to xargs and chmod. If you want to see an example without changing any permissions, just substitute ls -l like so…

find /home/user -name '*.ksh' | xargs ls -l

How do I send email from the linux command line?

Tue, 15 Oct 2013 17:04:19 +0000

Need to email from your linux command line? Assuming the underlying pieces are configured correctly (run the 1st example to see) here are the basics…

Send a test email

mail -s "Hi there" someone@somewhere.com

Include some body text…

echo "This is some body text" | mail -s "Hi there" someone@somewhere.com

Email the contents of a file to someone…

mail -s "My Subject" someone@somewhere.com < /path/to/your/file.txt

You can [find more here][1][1]: http://www.simplehelp.net/2008/12/01/how-to-send-email-from-the-linux-command-line/

What timezone is my linux server configured for?

Thu, 19 Sep 2013 07:16:09 +0000

To find out, just run the following command from the shell…

date +%Z

Setting up the ultimate console

Fri, 15 Feb 2013 15:40:00 +0000

If you’ve been coding for any time, chances are you’ve become acquainted with the command line. While some love it and some hate it I think it’s wonderful for getting things done quickly. Unfortunately, the windows command line is crap. With that said I finally got fed up and asked that most important programmer question of “How do I get a better command line”?

That search led me to an open source application called appropriately enough: Console. When I first opened it up and started playing around I have to admit my first reaction was “meh”. Then I asked that import question again, “How can this be made better”? A little Googling and experimentation later I got things working to my satisfaction.

So, here for your one-stop quick-reference is how to setup the ultimate console for Windows…

Assumptions

You access the console for all different platforms (not just Windows)
You already have putty and its utilities downloaded – and are familiar with their use
You already have installed cygwin – and are familiar with its use

Installation

[Download Console][1] and extract the contents to the folder of your choice
[Download ANSICON][2] and extract the following files to the same directory you extracted Console to.
- ANSI32.dll
- ANSI64.dll
- ansicon.exe

Great, now you have Console installed, but this is the configuration I use to make it really rock.

Configuration

These can all be found under “Edit -> Settings”

Appearance
- Custom color: Change it to an awsome green
Appearance -> More…
- Uncheck “Show toolbar”
- Select “Alpha” under “Window transparency” and set “Active window” to 25 and “Inactive window” to 40
Behavior
- Check “Copy on select”
- Uncheck “Clear selection on copy”
Hotkeys
- Set “New Tab 1” to “Ctrl+T”
- Set “Copy selection” to “Ctrl+C”
- Set “Paste” to “Ctrl+V”
Hotkeys -> Mouse
- Set “Copy/clear selection” to “None”
- Set “Select text” to “Left”
- Set “Paste text” to “Right”
- Set “Context menu” to “Middle”

Tab Configuration

Now that you have your console looking pretty and being functional, lets hook it into putty and cygwin. You can do this configuration under “Edit -> Settings -> Tabs”. Obviously, the directories you’ve chosen for things will be different (change to your values).

Cygwin: Add a tab and set the shell to:

C:\cygwin\bin\bash.exe --login -i

Putty: Add a tab and set the shell to:

C:\apps\Console2\ansicon.exe "C:\apps\Putty\plink.exe" -load "aSavedSessionName"

Reference

[Console Project][1]
[A Better Windows Command Prompt][3]
[Use PuTTY with Console2][4]
[Configuring Console2 with Cygwin][5]

[1]: http://console.sourceforge.net/ [2]: https://github.com/adoxa/ansicon/downloads [3]: http://www.hanselman.com/blog/Console2ABetterWindowsCommandPrompt.aspx [4]: http://blog.jimueller.com/post/29709142253/use-putty-with-console2 [5]: http://blog.quibb.org/2011/11/configuring-console2-with-cygwin/

How to list all environment variables

Thu, 14 Feb 2013 03:47:23 +0000

Often times you can find useful information in your environment variables, but since it’s something you don’t have to do everyday it’s easy to forget. Here’s a refresher:

Windows
First bring up a command prompt, then run the following…

set

To see the value of a single environment variable:

echo %ENVVAR%

Linux
Run the following from a terminal to see all environment variables…

printenv

To see the value of a single environment variable:

echo $ENVVAR

Setup password-less SSH and SCP with public/private keys

Sun, 03 Feb 2013 07:25:11 +0000

Want to scp a file to another server without having to enter the password? Want to just make your security even stronger? Public/private keys to the rescue! Of course, if you don’t know what I’m talking about or why you would want to do this, feel free to google it or just go visit another site.

For those still with me, you need access to both the local and remote servers (duh). I’m going to refer to the server/host you are logged into as the local machine, and the one you want to connect to as the remote machine.

First, on the local machine you need to generate your public and private keys. To do so, enter this command…

ssh-keygen -t rsa

Be sure to just hit enter to the questions you’re prompted with (otherwise you will have to enter a password when connecting with the keys – which goes against the whole point of this post). This will create a couple of files in your .ssh directory (something like id_rsa and id_rsa.pub – your private and public keys respectively). Your public key (the file you want to distribute) ends in “.pub”. Assuming you have that file, send it over to the remote machine (perhaps with scp, yes?)…

scp ./id_rsa.pub user@server.com:/home/user/.ssh/id_rsa.pub

Of course password authentication isn’t enabled yet so you’ll have to enter the password. Next up, you need to login to the remote machine and visit your .ssh directory, and cat the .pub file into your “authorized_keys” file (don’t worry, the command below will create the file if it doesn’t exist. I leave it to you to know how to create the .ssh directory if needed)…

cat id_rsa.pub >> authorized_keys

It’s then a good idea to secure your file and delete the temporary public key on the remote machine…

chmod 600 authorized_keys
rm id_rsa.pub

Once you’ve done all this, you should now be able to connect without a password! Just use your private key…

ssh -i ./id_rsa user@server.com

Have fun!

SSH, SFTP, and SCP on non-standard ports

Sun, 03 Feb 2013 04:24:05 +0000

One common suggestion for securing a secure shell connection on Linux is to change the port that ssh runs on. But how exactly do you connect to a server on one of these non-standard ports?

Naturally, the syntax is different for each so here goes:

ssh

ssh -p 33432 user@server.com

Yes, ssh uses a lower case p command line argument. Just wait until we get to scp.

sftp

sftp -o "Port 33432" user@server.com

Of course sftp uses a plain English option parameter of “Port XXXX”

scp

scp -P 33432 /home/user/file.txt user@server.com:/home/user

And naturally scp uses a capital P

Gotta love consistency!

Mount a Windows Share in Linux

Fri, 25 Jan 2013 17:23:54 +0000

So you want to access a Windows UNC share in Linux? No problem if you know the magic commands and have root access. This can be a great alternative to running cygwin if you are able to access a linux box (directly or via a virtual machine).

First, start by creating your mount point…

sudo mkdir /mnt/share1

Then, actually mount your UNC share to the mount point…

sudo mount -t cifs "//server/share1" /mnt/share -o username=my,password=secret

This would make \server\share1 available to you on the Linux box as /mnt/share. Note the use of the options parameter (-o) to specify username and password.

Execute a SQL file via the MySQL command line in one line

Wed, 12 Dec 2012 03:04:40 +0000

Have a large script (like a DB dump) you want to execute from the command line in one line so you can just drop it in a batch/shell script and run with it? Assuming MySql is on your system path, here’s the command…

mysql -u user -ppass < C:\temp\myscript.sql

Where “user” is the username, and “pass” is the password (note, there’s no space or other delimiter between the “-p” and the password)

Add a SSL Cert to Your Java Keystore

Thu, 06 Dec 2012 19:51:19 +0000

If you’ve ever had to add a SSL certificate to a java keystore, you know that the command is a little convoluted. Here for your and my reference is the command…

 "D:\Program Files\Java\jre1.5.0_22\bin\keytool" -import -trustcacerts -alias MyCA2 -file C:\MyCA2.crt -keystore "D:\Program Files\Java\jre1.5.0_22\lib\security\cacerts"

When promoted for the password, the default is “changeit”, and make sure to choose/type “yes” when asked if you want to trust the cert.