Linux on MarkJacobsen.net

Use SCP to copy files between remote and local hosts

Tue, 05 Feb 2019 01:44:54 +0000

So you need to copy a file you have on a remote server locally, or send a local file to a remote server? Fortunatally, it’s pretty easy with scp. Here’s an example to copy a remote server to the local machine.

scp -P 65124 remoteuser@remotehost:/var/lib/path/to/file.txt /var/lib/path/to/

…which will connect to “remotehost” on port 65124 as “remoteuser” and copy the /var/lib/path/to/file.txt on “remotehost” to the local /var/lib/path/to/ directory.

Need to copy a local file to a remote host? Just switch the parameters up!

scp -P 65124 /var/lib/path/to/file.txt remoteuser@remotehost:/var/lib/path/to/

Set a proxy and carry over the values to sudo

Fri, 25 Jan 2019 16:15:51 +0000

If you’ve ever been behind an enterprise proxy server and needed to reach things on the outside, you probably know how frustrating it is to try and just use your normal commands for something as simple as a wget. So, to get started you will need to know the URL and port for your proxy server, but once you have that, start by editing your .bashrc and add the following (setting proxyhost, the port, and adjusting “no_proxy” as appropriate)…

function proxyon(){
 echo -n "password (for proxy): "
 read -es password
 export http_proxy="http://$USER:$password@proxyhost.com:8181/"
 export HTTP_PROXY=$http_proxy
 export https_proxy=$http_proxy
 export HTTPS_PROXY=$http_proxy
 export ftp_proxy=$http_proxy
 export rsync_proxy=$http_proxy
 export no_proxy="localhost,127.0.0.1,localaddress,.localdomain.com,.domain.com"
 echo -e "\nProxy environment variable set."
}
function proxyoff(){
 unset HTTP_PROXY
 unset http_proxy
 unset HTTPS_PROXY
 unset https_proxy
 unset FTP_PROXY
 unset ftp_proxy
 unset RSYNC_PROXY
 unset rsync_proxy
 echo -e "\nProxy environment variable removed."
}

# remove next line if you don't want to set proxy automatically on login
proxyon

Now you can turn your proxy on and off with the “proxyon” and “proxyoff” commands, but that doesn’t help you when you try to install something with yum or apt that requires sudo. To carry over the proxy values, you’ll need to “sudo visudo” and add the “Defaults” line like shown below…

###############################################
# DO NOT ADD ANYTHING FROM HERE TO END OF FILE #
#
#includedir /etc/sudoers.d
#
# DO NOT ADD ANYTHING ABOVE THIS LINE###########
Defaults env_keep = "http_proxy https_proxy HTTP_PROXY HTTPS_PROXY ftp_proxy"

… and now, hopefully you are all set.

Configure a Static IP in Ubuntu 18.04

Mon, 21 Jan 2019 23:00:56 +0000

As with all things Linux there’s a million ways to do things, and every brilliant release comes up with a new way to do the same old things in a “better” way. Can you tell I’m be sarcastic yet? Good.

Anyway, after spending over an hour trying to get a freakin static IP on my desktop box I thought I’d share in case it saved anyone hours of their life.

My simple goal was to provide a static IP. Nothing more. Nothing less. Well, the “better” way of doing it now appears to be with “netplan”, and unless you enjoy banging your head against the wall, just create/edit this file and replace with the network interface (enp3s1), IP (192.168.20.55) and gateway (192.168.20.1) of your choice (this config appears to use the DNS servers from the router w/ DHCP turned on). Oh yeah, and don’t bother with the GUI based configuration tool, because apparently they’re not connected at all, and using this will force the config from this file (you’ve been warned and saved hours)

sudo nano /etc/netplan/01-networkd.yaml

# this is the actual file content
network:
 version: 2
 renderer: networkd
 ethernets:
 enp3s1:
 dhcp: no
 addresses: [192.168.20.55/24]
 gateway4: 192.168.20.1

Using sed to view and replace text on a specific file in Linux

Sat, 17 Mar 2018 14:26:00 +0000

Ever need to replace very specific text on a particular line in a very large file on Linux? Enter the sed command.

To view the line (let’s say line 864 in this case) in the specific file, issue this command…

$ sed -n '864'p myfile.txt

And let’s say you get back the string ” I hate cheese”. Since we all know that’s not true, you can use a different variation of the sed command to replace “hate” with “love” like so…

$ sed -i '864s/hate/love/' myfile.txt

And if you’re really curious on how this even came about, it was due to an error in the phpMyAdmin export.php script when you had to change “break 2” to “break” on line 846 as identified here: https://askubuntu.com/questions/928883/500-error-on-phpmyadmin-export-on-phpmyadmin-export-php/930975

Linux Startup Email

Sat, 21 Oct 2017 21:59:00 +0000

Want to receive an email any time your linux server reboots? Just create this script…

sudo nano /usr/local/sbin/startupemail.sh

Here’s the code for the script…

#!/bin/sh

# Lets things startup
sleep 60

# Get info
IP="$(ifconfig | grep -A 1 'eth0' | tail -1 | cut -d ':' -f 2 | cut -d ' ' -f 1)"
HOSTNAME=`hostname`

#Send email
echo "IP: $IP, Host: $HOSTNAME" | mail -s "Server Started" me@there.com

Make sure to make it executable…

sudo chmod u+x /usr/local/sbin/startupemail.sh

Then add it to your /etc/rc.local file

OwnCloud webapp not reflecting true list of files

Mon, 06 Jun 2016 18:54:00 +0000

If you’re using OwnCloud and have a process whereby your add or remove files from directories via some sort of server process, you may find that you need to force OwnCloud to update/rescan the file listing. To do so you can manually run the following command…

sudo -u username php /path/to/owncloud/console.php files:scan --all

You may also want to consider adding it to the crontab for “username” on a daily basis.

Thanks to this site for pointing me in the right direction.

Sendmail isn’t sending to domain.com : Linux

Thu, 02 Jun 2016 12:31:00 +0000

Say you’re using sendmail to relay email on a web server that you own that’s named “domain.com”, and email is being delivered fine to any domain except domain.com. What could be the problem?

Rename your server. No server should be named with an actual domain name.
Change /etc/hostname to the new name for your server
Update /etc/hosts to remove the domain name and replace with the new hostname
Remove the domain name from /etc/mail/local-host-names
Restart sendmail “sudo service sendmail restart”
Restart your server “sudo reboot”

Once you’ve done that test sendmail out like so…

echo "command line test" | mail -s "Sendmail test" me@domain.com

Improved Linux command history

Sun, 03 Apr 2016 23:14:00 +0000

With four lines setup one time you can type the beginning of a command (ex: “ls”) then press the up and down arrows to see the most recent commands containing the text you typed (ex: “ls ~/scripts”) making it much easier to find what you’re looking for. To make this happen, create a text file called .inputrc in your home folder and put the following four lines inside:

"\e[A": history-search-backward
"\e[B": history-search-forward
set show-all-if-ambiguous on
set completion-ignore-case on

Linux ls output colors

Mon, 23 Mar 2015 15:48:00 +0000

If you’ve used linux much and your terminal is set to display files and directories in color, you know how frustrating it can be to have dark blue text on a black background like so (I’ve seen it much worse too)…

[][1]

If you would like to modify the output so the colors are a bit more readable, you can add the following to your .bashrc file…

alias ls='ls --color'
LS_COLORS='di=33:fi=36:ex=31'
export LS_COLORS

which will produce an easier to read version like so (with directories yellow, files a crayon/teal, and executable files red)…

[][2]

As pointed out in this article…

The first line makes ls use the –color parameter by default, which tells ls to display files in different colours based on the setting of the LS_COLORS variable.

The second line is the tricky one, and what I have worked out so far has been by trial and error. The parameters (di, fi, etc.) refer to different Linux file types. I have worked them out as shown

di = directory
fi = file
ln = symbolic link
pi = fifo file
so = socket file
bd = block (buffered) special file
cd = character (unbuffered) special file
or = symbolic link pointing to a non-existent file (orphan)
mi = non-existent file pointed to by a symbolic link (visible when you type ls -l)
ex = file which is executable (ie. has ‘x’ set in permissions).

The *.rpm=90 parameter at the end tells ls to display any files ending in .rpm in the specified colour, in this case colour 90 (dark grey). This can be applied to any types of files (eg. you could use ‘*.png=35’ to make jpeg files appear purple.) As many or as few parameters as you like can go into the LS_COLORS variable, as long as the parameters are separated by colons.

Using trial and error (and a little bash script I wrote… my first one ever! 🙂 I worked out all the colour codes, at least my interpretation of them –

0 = default colour
1 = bold
4 = underlined
5 = flashing text
7 = reverse field
31 = red
32 = green
33 = orange
34 = blue
35 = purple
36 = cyan
37 = grey
40 = black background
41 = red background
42 = green background
43 = orange background
44 = blue background
45 = purple background
46 = cyan background
47 = grey background
90 = dark grey
91 = light red
92 = light green
93 = yellow
94 = light blue
95 = light purple
96 = turquoise
100 = dark grey background
101 = light red background
102 = light green background
103 = yellow background
104 = light blue background
105 = light purple background
106 = turquoise background[1]: https://i0.wp.com/markjacobsen.net/wp-content/uploads/2015/03/dirColorBefore.png [2]: https://i0.wp.com/markjacobsen.net/wp-content/uploads/2015/03/dirColorAfter.png

Pass a Command Line Argument to an Alias

Fri, 28 Nov 2014 15:11:06 +0000

Ever want to pass in a command line argument to an alias? You would think you could just do it with $1, but actually you have to create a function and then call that function. So, for instance if you want to pass a portion of a log file name, you could set your alias like this…

alias catlog='function _catIt() { cat /var/logs/$1.log; };_catIt'

then, when you want to see the XX00D log, you just call your alias like so…

catlog XX00D

Checking and Maintaining Linux Disk Space

Mon, 10 Nov 2014 13:30:00 +0000

Ever need to find out what’s using the most space on your Linux box? There are a couple commands that will help make things easier…

To find out how much space you have use:

df -h .

Which will give you output like so…

Filesystem Size Used Avail Use% Mounted on
/dev/sda 20G 15G 4.2G 79% /

To list out the directories using the most space, use this handy command…

sudo du /usr/local | sort -n

… where /usr/local is the directory you want info for (you can also just start from root: /)

Learning VIM

Wed, 24 Sep 2014 11:49:00 +0000

So you would think that as a programmer for many moons now that I would have considerable experience in VIM and VI, but I’m hear today to admit that I am not. For years I’ve gotten by with my favorite Windows text editor notepad++, but the geek in me wanted to know why everyone loves Vim so much. To that end the 2 things I started with are…

Downloading the Windows vim editor from [Vim.org][1]

Working through the tutorial at http://www.openvim.com/tutorial.html

I can’t say I’m a convert yet, but it’s always nice to have another tool in the toolbelt. Here for your and my reference is [my quick VIM cheat sheet][2].[1]: http://www.vim.org [2]: http://visit.markjacobsen.net/vim-cheat-sheet/ “VIM Cheat Sheet”

UNIX: Don’t Show Permission Denied Errors when using find Command

Wed, 10 Sep 2014 11:49:00 +0000

If you’re using the unix find command to search for files with a particular name

find -name theName
find: `./dir/thing': Permission denied
find: `./dir/thing1': Permission denied
find: `./dir/thing2': Permission denied

You might get distracted by all those “Permission denied” errors. The easy way to solve this is to redirect stderr to /dev/null like so…

find -name theName 2>/dev/null

UNIX: No such file or directory but the file exists

Fri, 29 Aug 2014 10:49:00 +0000

Ever have a script that is executing another script and get an error that looks like this?…

/path/stub.ksh[2]: /path/XX/script.ksh: not found [No such file or directory]

Then you make sure that the file does in fact exist, and that you can read it?

So why does it say there’s “No such file or directory”?

Might want to check if the file has Windows line breaks. Easiest way to do that is run the following…

cat -v /path/XX/script.ksh

… and you’ll probably see that your lines end with ^M characters. If so, you have Window’s line breaks and you might want to look into not introducing them in the first place (save in Unix format), or check out the dos2unix command.

Quick apt-get Tutorial

Thu, 10 Apr 2014 07:18:38 +0000

List installed packages…

dpkg --get-selections | grep -v deinstall

List installed packages (but filter based on name)…

dpkg --get-selections | grep -v deinstall | grep filter

Install a package…

sudo apt-get install the-package-name

Uninstall/remove a package…

sudo apt-get remove the-package-name

Update package lists/dependencies (does not actually install anything)…

sudo apt-get update

Apply updates to existing packages based on an “update” call (without removing anything)…

sudo apt-get upgrade

Apply updates to existing packages based on an “update” call and remove obsolete packages…

sudo apt-get dist-upgrade

Quick CRON Tutorial

Wed, 02 Apr 2014 02:50:16 +0000

The fastest way to setup scheduled tasks in Linux is with cron. To list your cron tasks, run:

sudo crontab -l

To modify your cron tasks, run:

sudo crontab -e

For help generating the string for when your tasks should run, just google around for “cron calculator”, and remember to test, test, test.

What files are taking up the most space on my LINUX box?

Tue, 28 Jan 2014 20:42:50 +0000

Use du piped to sort for a nice list with the files using the most space at the bottom…

cd /dir/you/care/about
du -a|sort -n

Change permissions on files of a specific type in linux

Thu, 17 Oct 2013 07:09:32 +0000

Need to change all *.ksh files to be executable under a directory, but not having luck with a recursive chmod? The issue is you need to combine chmod with a find and xargs like so…

find /home/user -name '*.ksh' | xargs chmod 744

The first piece lists all the files under the path that match *.ksh and passes them to xargs and chmod. If you want to see an example without changing any permissions, just substitute ls -l like so…

find /home/user -name '*.ksh' | xargs ls -l

How do I send email from the linux command line?

Tue, 15 Oct 2013 17:04:19 +0000

Need to email from your linux command line? Assuming the underlying pieces are configured correctly (run the 1st example to see) here are the basics…

Send a test email

mail -s "Hi there" someone@somewhere.com

Include some body text…

echo "This is some body text" | mail -s "Hi there" someone@somewhere.com

Email the contents of a file to someone…

mail -s "My Subject" someone@somewhere.com < /path/to/your/file.txt

You can [find more here][1][1]: http://www.simplehelp.net/2008/12/01/how-to-send-email-from-the-linux-command-line/

What timezone is my linux server configured for?

Thu, 19 Sep 2013 07:16:09 +0000

To find out, just run the following command from the shell…

date +%Z

How do I run a script in the background?

Tue, 17 Sep 2013 07:14:07 +0000

Use nohup! Here’s an example…

nohup /some/script.ksh

And even better, to redirect all output to a specific location…

nohup /home/user/script.ksh > /home/user/script.log 2>&1 &

Yes, the ” 2>&1 &” at the end is important.

Setting up the ultimate console

Fri, 15 Feb 2013 15:40:00 +0000

If you’ve been coding for any time, chances are you’ve become acquainted with the command line. While some love it and some hate it I think it’s wonderful for getting things done quickly. Unfortunately, the windows command line is crap. With that said I finally got fed up and asked that most important programmer question of “How do I get a better command line”?

That search led me to an open source application called appropriately enough: Console. When I first opened it up and started playing around I have to admit my first reaction was “meh”. Then I asked that import question again, “How can this be made better”? A little Googling and experimentation later I got things working to my satisfaction.

So, here for your one-stop quick-reference is how to setup the ultimate console for Windows…

Assumptions

You access the console for all different platforms (not just Windows)
You already have putty and its utilities downloaded – and are familiar with their use
You already have installed cygwin – and are familiar with its use

Installation

[Download Console][1] and extract the contents to the folder of your choice
[Download ANSICON][2] and extract the following files to the same directory you extracted Console to.
- ANSI32.dll
- ANSI64.dll
- ansicon.exe

Great, now you have Console installed, but this is the configuration I use to make it really rock.

Configuration

These can all be found under “Edit -> Settings”

Appearance
- Custom color: Change it to an awsome green
Appearance -> More…
- Uncheck “Show toolbar”
- Select “Alpha” under “Window transparency” and set “Active window” to 25 and “Inactive window” to 40
Behavior
- Check “Copy on select”
- Uncheck “Clear selection on copy”
Hotkeys
- Set “New Tab 1” to “Ctrl+T”
- Set “Copy selection” to “Ctrl+C”
- Set “Paste” to “Ctrl+V”
Hotkeys -> Mouse
- Set “Copy/clear selection” to “None”
- Set “Select text” to “Left”
- Set “Paste text” to “Right”
- Set “Context menu” to “Middle”

Tab Configuration

Now that you have your console looking pretty and being functional, lets hook it into putty and cygwin. You can do this configuration under “Edit -> Settings -> Tabs”. Obviously, the directories you’ve chosen for things will be different (change to your values).

Cygwin: Add a tab and set the shell to:

C:\cygwin\bin\bash.exe --login -i

Putty: Add a tab and set the shell to:

C:\apps\Console2\ansicon.exe "C:\apps\Putty\plink.exe" -load "aSavedSessionName"

Reference

[Console Project][1]
[A Better Windows Command Prompt][3]
[Use PuTTY with Console2][4]
[Configuring Console2 with Cygwin][5]

[1]: http://console.sourceforge.net/ [2]: https://github.com/adoxa/ansicon/downloads [3]: http://www.hanselman.com/blog/Console2ABetterWindowsCommandPrompt.aspx [4]: http://blog.jimueller.com/post/29709142253/use-putty-with-console2 [5]: http://blog.quibb.org/2011/11/configuring-console2-with-cygwin/

How to list all environment variables

Thu, 14 Feb 2013 03:47:23 +0000

Often times you can find useful information in your environment variables, but since it’s something you don’t have to do everyday it’s easy to forget. Here’s a refresher:

Windows
First bring up a command prompt, then run the following…

set

To see the value of a single environment variable:

echo %ENVVAR%

Linux
Run the following from a terminal to see all environment variables…

printenv

To see the value of a single environment variable:

echo $ENVVAR

Setup password-less SSH and SCP with public/private keys

Sun, 03 Feb 2013 07:25:11 +0000

Want to scp a file to another server without having to enter the password? Want to just make your security even stronger? Public/private keys to the rescue! Of course, if you don’t know what I’m talking about or why you would want to do this, feel free to google it or just go visit another site.

For those still with me, you need access to both the local and remote servers (duh). I’m going to refer to the server/host you are logged into as the local machine, and the one you want to connect to as the remote machine.

First, on the local machine you need to generate your public and private keys. To do so, enter this command…

ssh-keygen -t rsa

Be sure to just hit enter to the questions you’re prompted with (otherwise you will have to enter a password when connecting with the keys – which goes against the whole point of this post). This will create a couple of files in your .ssh directory (something like id_rsa and id_rsa.pub – your private and public keys respectively). Your public key (the file you want to distribute) ends in “.pub”. Assuming you have that file, send it over to the remote machine (perhaps with scp, yes?)…

scp ./id_rsa.pub user@server.com:/home/user/.ssh/id_rsa.pub

Of course password authentication isn’t enabled yet so you’ll have to enter the password. Next up, you need to login to the remote machine and visit your .ssh directory, and cat the .pub file into your “authorized_keys” file (don’t worry, the command below will create the file if it doesn’t exist. I leave it to you to know how to create the .ssh directory if needed)…

cat id_rsa.pub >> authorized_keys

It’s then a good idea to secure your file and delete the temporary public key on the remote machine…

chmod 600 authorized_keys
rm id_rsa.pub

Once you’ve done all this, you should now be able to connect without a password! Just use your private key…

ssh -i ./id_rsa user@server.com

Have fun!

SSH, SFTP, and SCP on non-standard ports

Sun, 03 Feb 2013 04:24:05 +0000

One common suggestion for securing a secure shell connection on Linux is to change the port that ssh runs on. But how exactly do you connect to a server on one of these non-standard ports?

Naturally, the syntax is different for each so here goes:

ssh

ssh -p 33432 user@server.com

Yes, ssh uses a lower case p command line argument. Just wait until we get to scp.

sftp

sftp -o "Port 33432" user@server.com

Of course sftp uses a plain English option parameter of “Port XXXX”

scp

scp -P 33432 /home/user/file.txt user@server.com:/home/user

And naturally scp uses a capital P

Gotta love consistency!

Mount a Windows Share in Linux

Fri, 25 Jan 2013 17:23:54 +0000

So you want to access a Windows UNC share in Linux? No problem if you know the magic commands and have root access. This can be a great alternative to running cygwin if you are able to access a linux box (directly or via a virtual machine).

First, start by creating your mount point…

sudo mkdir /mnt/share1

Then, actually mount your UNC share to the mount point…

sudo mount -t cifs "//server/share1" /mnt/share -o username=my,password=secret

This would make \server\share1 available to you on the Linux box as /mnt/share. Note the use of the options parameter (-o) to specify username and password.

Start ColdFusion8 on Ubuntu Server Start

Thu, 17 Jan 2013 04:41:50 +0000

Ok, so this is not the most bleeding edge post, but hopefully useful if you need it. Here’s what you need to do if you need to get ColdFusion8 to start on server reboot (since it doesn’t appear to install itself as a service) using Ubuntu server.

First, create a service script and make sure it’s executable like so…

sudo touch /etc/init.d/coldfusion
sudo chmod 755 /etc/init.d/coldfusion

Next, you need to actually put something in the script…

sudo nano /etc/init.d/coldfusion

Here’s what to put in the file…

#! /bin/sh

### BEGIN INIT INFO
# Short-Description: ColdFusion8 service
### END INIT INFO

file=/opt/coldfusion8/bin/coldfusion

case "$1" in
stop)
$file stop
;;
status)
$file status
;;
restart)
$file restart
;;
*)
$file start
;;
esac

exit 0

Finally, you need to tell Ubuntu to use your service and make it available

sudo update-rc.d coldfusion defaults