Archive

Archive for the ‘Uncategorized’ Category

Nagios failover setup

September 3rd, 2010

Once you have nagios configured you can setup a failover setup where if the nagios master is offline then a standby slave will enable notifications and checking. Here are the setup notes and as custome script i wrote in pythong to achieve this.

the following command checks if nagios is running locally

[root@scrappy nagios]# /usr/local/nagios/libexec/check_nagios -F /usr/local/nagios/var/status.dat -e 1 -C ‘/usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg’

NAGIOS OK: 1 process, status log updated 5 seconds ago

now, set in nrpe config as following on remote slave/master to verify is running. must restart nrpe on machine installed on to reread nrpe.cfg

command[check_nagios_failover]=/usr/local/nagios/libexec/check_nagios -F /usr/local/nagios/var/status.dat -e 1 -C ‘/usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg’

test nrpe remote command.

[root@scrappy nagios]# /usr/local/nagios/libexec/check_nrpe -H james -c check_nagios_failover

NAGIOS OK: 15 processes, status log updated 0 seconds ago

modify nagios.cfg on nagios slave. restart nagios.

execute_service_checks=0

enable_notifications=0

check_external_commands=1

now set crontab on slave to check for master failure.

[root@scrappy etc]# crontab -l

* * * * * nagios /usr/local/nagios/set_slave_status.py > /dev/null

Create script and modify master ip.


#!/usr/bin/python

import os

import commands

import sys

master='james'

slave='scrappy'

tmp_file='/tmp/nagios-failover-state.txt'

commandfile='/usr/local/nagios/var/rw/nagios.cmd'

now=commands.getoutput('date +%s')

email=’email@domain.com‘

out=commands.getoutput(’/usr/local/nagios/libexec/check_nrpe -H ‘+master+’ -c check_nagios_failover’)

#if not 0, then master is down.

master_return_val=out.find(’OK’)

#store state information

def failover_save_state(x):

fh=open(tmp_file,’w')

fh.write(x)

fh.close()

def sync_nagios_files():

os.system(’rsync -av ‘+master+’:/usr/local/nagios/etc/objects/* /usr/local/nagios/etc/objects/’)

os.system(’/etc/init.d/nagios reload’)

print “Master conf files synced with Slave.”

#read current state information

fh=open(tmp_file,’r')

current_state=fh.readlines()

fh.close()

current_state=current_state[0]

#enable or disable checks for hosts

#print current_state

if master_return_val <= 0 and current_state is not ‘enabled’:

if current_state==’enabled’:

print “Nagios slave (”+slave+”) is active. Nagios master (”+master+”) is down. No state change”

else:

os.system(’/usr/bin/printf “[%lu] ENABLE_NOTIFICATIONS\n” ‘+now+’ > ‘+commandfile)

os.system(’/usr/bin/printf “[%lu] START_EXECUTING_HOST_CHECKS\n” ‘+now+’ > ‘+commandfile)

os.system(’/usr/bin/printf “[%lu] START_EXECUTING_SVC_CHECKS\n” ‘+now+’ > ‘+commandfile)

os.system(’echo elvis has left the building | mail -s “[Nagios] Master Down! Slave Enabled.” ‘+email)

print “Nagios Failover enabled”

failover_save_state(’enabled’)

sync_nagios_files()

elif current_state is not ‘disabled’:

if current_state==’disabled’:

print “Nagios master (”+master+”) is enabled. No state change.”

else:

os.system(’/usr/bin/printf “[%lu] DISABLE_NOTIFICATIONS\n” ‘+now+’ > ‘+commandfile)

os.system(’/usr/bin/printf “[%lu] STOP_EXECUTING_HOST_CHECKS\n” ‘+now+’ > ‘+commandfile)

os.system(’/usr/bin/printf “[%lu] STOP_EXECUTING_SVC_CHECKS\n” ‘+now+’ > ‘+commandfile)

os.system(’echo We are all out of donuts. | mail -s “[Nagios] Master Restored. Slave Disabled.” ‘+email)

print “Nagios master restored”

failover_save_state(’disabled’)

sync_nagios_files()

make executable

 

[root@scrappy nagios]# chown nagios.nagios /usr/local/nagios/etc/set_slave_status.py

[root@scrappy etc]# chmod o=rwx set_slave_status.py

Uncategorized

Network copy tests

August 23rd, 2010

What is faster cp or rsync? Each test is done twice to verify consistency. Let’s find out…

Rsync over NFS

#donkey:~# time rsync -av /mnt/backupserv/10gb /data/10gb
sending incremental file list
10gb

sent 10738729030 bytes  received 31 bytes  41223528.07 bytes/sec
total size is 10737418240  speedup is 1.00

real    4m19.773s
user    1m31.566s
sys     2m23.025s
#donkey:~# time rsync /mnt/backupserv/10gb /data/10gb

real    4m15.026s
user    1m26.453s
sys     2m24.837s

RSYNC direct

#donkey:~# time rsync -av backupserv:/mnt/data/10gb /data/
receiving incremental file list
10gb

sent 30 bytes  received 10738729035 bytes  33401956.66 bytes/sec
total size is 10737418240  speedup is 1.00

real    5m21.004s
user    3m6.644s
sys     2m1.376s

#donkey:~# time rsync -av backupserv:/mnt/data/10gb /data/
receiving incremental file list
10gb

sent 30 bytes  received 10738729035 bytes  32790012.41 bytes/sec
total size is 10737418240  speedup is 1.00

real    5m26.857s
user    3m5.924s
sys     2m8.372s

CP over NFS

#donkey:~# time cp /mnt/backupserv/10gb /data/

real    2m17.176s
user    0m0.024s
sys     0m57.276s

#donkey:~# time cp /mnt/backupserv/10gb /data/

real    2m16.983s
user    0m0.020s
sys     0m57.920s

Uncategorized

Bacula: ‘query’ command doest work.

August 2nd, 2010

Need to move sample-query.sql into /etc/bacula.

*query
Automatically selected Catalog: MyCatalog
Using Catalog “MyCatalog”
Available queries:
1: List up to 20 places where a File is saved regardless of the directory
2: List where the most recent copies of a file are saved
3: List last 20 Full Backups for a Client
4: List all backups for a Client after a specified time
5: List all backups for a Client
6: List Volume Attributes for a selected Volume
7: List Volumes used by selected JobId
8: List Volumes to Restore All Files
9: List Pool Attributes for a selected Pool
10: List total files/bytes by Job
11: List total files/bytes by Volume
12: List Files for a selected JobId
13: List Jobs stored on a selected MediaId
14: List Jobs stored for a given Volume name
15: List Volumes Bacula thinks are in changer
16: List Volumes likely to need replacement from age or errors
17: List Volumes Bacula thinks are eligible for the changer
18: List Volumes by Volume:
19: List Volumes by Jobs:
20: List Volumes for a jobname:

Uncategorized

Using Tar for Backups

July 23rd, 2010

Update file that already exists in tar archive
[root@james ~]# tar -uvf ./test.tar updatefile
Append file that doesnt not exist to archive.
[root@james ~]# tar -rvf ./test.tar newfilesfile
Extract one file from archive.
[root@james ~]# tar -xvf ./test.tar somefile

 

Uncategorized

Working with ‘Apt’ in Ubuntu

July 10th, 2010

Search for a package

apt-cache search <phrase>

Install a package

apt-get install <package>

Upgrade kernel

apt-get dist-upgrade

Uncategorized

Using the ‘touch’ command

July 7th, 2010

Create a new file using touch and modify its access time.

Set the date to Feb 1st, 2009 @ 730am.

[root@stone backups]# touch testfile2 -t 200902010730 testfile2

Verify settings..
[root@stone backups]# ls -la | grep testfile2
-rw-r–r–  1 root katze          0 Feb  1  2009 testfile2

Uncategorized

UFW Firewall HOWTO

May 25th, 2010

Add rule

root@firefly:/data/music# ufw allow from 10.200.80.132 to any port 3689

Delete rule by line number.

root@firefly:/data/music# ufw delete 1
Deleting:
allow 3689
Proceed with operation (y|n)? y
Rule deleted


View current rules

root@firefly:/data/music# ufw status
Status: active

To                         Action      From
–                         ——      —-
22                         ALLOW       Anywhere
21                         ALLOW       Anywhere
3689                       ALLOW       10.200.80.132

Uncategorized

Python: simple salary calc script

May 21st, 2010

#!/usr/bin/python

def main():
salary1 = input(”Enter Salary #1: “)
salary2 = input(”Enter Salary #2: “)

# SET GLOBAL TO SHARE VARS
global x
global y

if(salary1 < salary2):
x=salary1
y=salary2
else:
y=salary1
x=salary2

def make_percent():
global total

total = 100-(float(x)/float(y)*100)
total=int(round(total,0))
print “Total Difference:”,total,”%”

main()
make_percent()

root@donkey:~/tech/core# python calc_number_diff.py
Enter Salary #1: 75
Enter Salary #2: 76
Total Difference: 1 %

Uncategorized

Working with LVM. Or as i like to call, super raid0!

March 16th, 2010

Install packages

apt-get install lvm2 dmsetup mdadm reiserfsprogs xfsprogs

Create partitions

fdisk /dev/sda

-create new partition (’n')
-change system partition type to lvm (’t')
-enter ‘8e’ for hex partition type
-write partition table to disk (’w')

Create physical volume

root@backupserv:~# pvcreate /dev/sdb1 /dev/sda2 /dev/sdc1
Can’t open /dev/sdb1 exclusively.  Mounted filesystem?
Wiping software RAID md superblock on /dev/sda2
Physical volume “/dev/sda2″ successfully created
Wiping software RAID md superblock on /dev/sdc1
Physical volume “/dev/sdc1″ successfully created

root@backupserv:~# pvdisplay
“/dev/sda2″ is a new physical volume of “903.57 GB”
— NEW Physical volume —
PV Name               /dev/sda2
VG Name
PV Size               903.57 GB
Allocatable           NO
PE Size (KByte)       0
Total PE              0
Free PE               0
Allocated PE          0
PV UUID               zsP1EO-SlwB-nryo-gEP7-fInc-hJc4-ytVpLX

“/dev/sdc1″ is a new physical volume of “931.51 GB”
— NEW Physical volume —
PV Name               /dev/sdc1
VG Name
PV Size               931.51 GB
Allocatable           NO
PE Size (KByte)       0
Total PE              0
Free PE               0
Allocated PE          0
PV UUID               V3RZ2b-FBsM-HoI6-aQED-ouu9-LpR1-m02VLO

Create a Volume Group
root@backupserv:~# vgcreate data /dev/sdc1 /dev/sda2
Volume group “data” successfully created

root@backupserv:~# vgdisplay
— Volume group —
VG Name               data
System ID
Format                lvm2
Metadata Areas        2
Metadata Sequence No  1
VG Access             read/write
VG Status             resizable
MAX LV                0
Cur LV                0
Open LV               0
Max PV                0
Cur PV                2
Act PV                2
VG Size               1.79 TB
PE Size               4.00 MB
Total PE              469780
Alloc PE / Size       0 / 0
Free  PE / Size       469780 / 1.79 TB
VG UUID               XJVXrA-m5xO-GARp-6WT0-JFlX-7ARj-jxJChV

root@backupserv:~# vgscan
Reading all physical volumes.  This may take a while…
Found volume group “data” using metadata type lvm2fdisk /dev/sda

Create Logical Volume

root@backupserv:~# lvcreate –name share1 –size 100G data
Logical volume “share1″ created
root@backupserv:~# lvcreate –name share2 –size 500G data
Logical volume “share2″ created

*run lvscan for new logical volume scan.

Format partition

root@backupserv:~# mkfs.ext3 /dev/data/share1
mke2fs 1.41.9 (22-Aug-2009)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
6553600 inodes, 26214400 blocks
1310720 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=0
800 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872

Writing inode tables: 192/800

Mount device

root@backupserv:~# mkdir /mnt/share2
root@backupserv:~# mount /dev/data/share2 /mnt/share2
/dev/mapper/data-share2
516061624    202652 489644572   1% /mnt/share2

Deleting an Volume Group

root@backupserv:~# vgremove data
Do you really want to remove volume group “data” containing 2 logical volumes? [y/n]: yes
Can’t remove open logical volume “share1″
root@backupserv:~# umount /mnt/share*
root@backupserv:~# vgremove data
Do you really want to remove volume group “data” containing 2 logical volumes? [y/n]: yes
Do you really want to remove active logical volume “share1″? [y/n]: yes
Logical volume “share1″ successfully removed
Do you really want to remove active logical volume “share2″? [y/n]: yes
Logical volume “share2″ successfully removed
Volume group “data” successfully removed

Delete Physical Volume

root@backupserv:~# pvremove /dev/sdc1 /dev/sdb1
Labels on physical volume “/dev/sdc1″ successfully wiped
Physical Volume /dev/sdb1 not found

ref: http://www.howtoforge.com/linux_lvm_p2

Adding Additional Disk to LVM

create fdisk LVM partition first….

root@backupserv:~# pvcreate /dev/sdb1
Physical volume “/dev/sdb1″ successfully created

root@backupserv:~# vgextend vol1 /dev/sdb1
Volume group “vol1″ successfully extended
root@backupserv:~# vgdisplay vol1
— Volume group —
VG Name               vol1
System ID
Format                lvm2
Metadata Areas        3
Metadata Sequence No  5
VG Access             read/write
VG Status             resizable
MAX LV                0
Cur LV                1
Open LV               1
Max PV                0
Cur PV                3
Act PV                3
VG Size               2.70 TB
PE Size               4.00 MB
Total PE              708246
Alloc PE / Size       469238 / 1.79 TB
Free  PE / Size       239008 / 933.62 GB
VG UUID               RS2K3i-SOJ8-a230-fcQL-8PX0-Bpuj-qSUYKz

root@backupserv:~# lvextend -L +930G /dev/vol1/data1 /dev/sdb1
Extending logical volume data1 to 2.70 TB
Logical volume data1 successfully resized

root@backupserv:~# resize2fs /dev/vol1/data1
resize2fs 1.41.9 (22-Aug-2009)
Filesystem at /dev/vol1/data1 is mounted on /mnt/data; on-line resizing required
old desc_blocks = 115, new_desc_blocks = 173
….

*watch more space become available with ‘watch df -h’

Linux, Raid, Uncategorized

Find: xargs -vs- exec

March 16th, 2010

‘-exec’ using trailing ‘\’

[root@monk ~]# tar -cvf test.tar install.log
install.log
[root@monk ~]# time find /tmp/conf-2-xp -name *.xad -exec tar –append –file=/root/test.tar {} +;
find: /tmp/conf-2-xp/9a2f6257887d6e51f58ce2/amd64: Permission denied
find: /tmp/conf-2-xp/9a2f6257887d6e51f58ce2/i386: Permission denied
find: /tmp/conf-2-xp/System Volume Information: Permission denied
tar: Removing leading `/’ from member names

real    0m15.239s
user    0m0.464s
sys     0m2.628s
[root@monk ~]# du -h test.tar
310M    test.tar
[root@monk ~]# rm -f test.tar

*delete fiel and recreate since we are appending and comparing file size after each run.

‘-exec’ using trailing ‘+’

[root@monk ~]# tar -cvf test.tar install.log
install.log
[root@monk ~]# time find /tmp/conf-2-xp -name *.xad -exec tar –append –file=/root/test.tar {} \;
find: /tmp/conf-2-xp/9a2f6257887d6e51f58ce2/amd64: Permission denied
find: /tmp/conf-2-xp/9a2f6257887d6e51f58ce2/i386: Permission denied

real    0m39.015s
user    0m2.739s
sys     0m21.893s
[root@monk ~]# du -h test.tar
310M    test.tar

Using xargs.

*use -print0 option to prevent whitespace issue. ‘xargs -0′ passes first argument.

[root@monk ~]# rm test.tar
rm: remove regular file `test.tar’? yes
[root@monk ~]# tar -cvf test.tar install.log
install.log
[root@monk ~]# time find /tmp/conf-2-xp -name ‘*.xad’ -print0 | xargs -0 tar –append –file=/root/test.tar;
find: /tmp/conf-2-xp/9a2f6257887d6e51f58ce2/amd64: Permission denied
find: /tmp/conf-2-xp/9a2f6257887d6e51f58ce2/i386: Permission denied
find: /tmp/conf-2-xp/System Volume Information: Permission denied
tar: Removing leading `/’ from member names

real    0m15.073s
user    0m0.446s
sys     0m2.774s
[root@monk ~]# du -h test.tar
310M    test.tar

Result: Using ‘+’ is two times faster than using ‘\’. -exec and xargs are about the same speed, when using ‘+’ for -exec.

Uncategorized

Encrypting files with GPG (Gnu Private Guard)

February 3rd, 2010

-Create a text file with a secret phrase..

jesterj@jesterj-laptop:~$ cat > secret
donkey

-Encrypt file with password:

jesterj@jesterj-laptop:~$ gpg -c secret

-Verify file is encrypted:

jesterj@jesterj-laptop:~$ more secret.gpg
���(�U    �`�#.���ؖ-�$U

-Send to a different system, and test the file.

jesterj@jesterj-laptop:~$ scp secret.gpg jesterj@remotesystem:~
secret.gpg                                    100%   52     0.1KB/s   00:00

-Decrypt the file and enter passphrase.

jesterj@donkey:~$ gpg secret.gpg
gpg: CAST5 encrypted data
Enter passphrase:

-File is decrompressed:

jesterj@donkey:~$ more secret
donkey

Done!

soure: http://www.cyberciti.biz/tips/linux-how-to-encrypt-and-decrypt-files-with-a-password.html

Uncategorized

Verifying file integrity with md5sum

February 3rd, 2010

To check a files checksum simply run the following command:

jesterj@jesterj-laptop:~$ md5sum file1
f5fa31b4e964cc2a86140bc2a2e11a13  file1

This does a check of the contents of the file and compares to the 128 bit key located in the header of the file.

To check a group of files then you can do the following.

jesterj@jesterj-laptop:~$ md5sum password* > md5checksums.txt
jesterj@jesterj-laptop:~$ md5sum password*
f5fa31b4e964cc2a86140bc2a2e11a13  password
67c2a19b0fb22552a5aa7cdc6ede2634  password.gpg
f5fa31b4e964cc2a86140bc2a2e11a13  password.orig
11b4f8cd65095066a644139500309bad  passwords
3295d069764198d327edeba43f2b301c  passwords.gpg
jesterj@jesterj-laptop:~$ md5sum -c md5checksums.txt
password: OK
password.gpg: OK
password.orig: OK
passwords: OK
passwords.gpg: OK

source:
http://linux.byexamples.com/archives/198/md5-checksum-how-to/

Uncategorized

Disabling service on startup in ubuntu

January 26th, 2010

Need to install debian-helper-scripts.

root@pse07:/var/lib# service apparmor off
The program 'service' can be found in the following packages:
* debian-helper-scripts
* sysvconfig
Try: apt-get install <selected package>
bash: service: command not found
root@pse07:/var/lib# sudo apt-get install debian-helper-script
root@pse07:/var/lib# service apparmor stop
Unloading AppArmor profiles : done.

Uncategorized

Changing data dir for mysql & dealing with Apparmor

January 26th, 2010

In ubuntu there is program called apparmor that verifies certain programs are configured appropriately, mysql included. Just recently the partition that holds all the data (/var/lib/mysql) filed up. Since I was at 100% and this is a production box I needed to fix this quickly. My options were as follows:

1. Setup a sym link from /var/lib/mysql to /home/mysql, move data here OR
2. Simply create a new dir call /home/mysql, move data here

To do this edit /etc/mysql/my.cnf
...
datadir =/home/mysql
#datadir = /var/lib/mysql
...

Save the file and restart mysql and you will get the following error:

root@slave:/etc/mysql# /etc/init.d/mysql2 start
* Starting MySQL database server mysqld [fail]

The problem here is that apparmor is dissallowing mysql from running in any other location than /var/lib/mysql. You will need to edit the apparmor file for mysql.

root@slave:/etc/apparmor.d# grep "/var/lib/mysql" usr.sbin.mysqld
/var/lib/mysql/ r,
/var/lib/mysql/** rwk,

To correct this problem replace all instances of /var/lib/mysql with /home/mysql. Open the file with file and do a search and replace. See that I am escaping the slashes!

:%s /\/var\/lib\/mysql/\/home\/mysql/

Reload appamor and mysql should start up with the new config in your my.cnf file!

root@slave:/etc/apparmor.d# /etc/init.d/apparmor restart
Reloading AppArmor profiles : done.
root@slave:/etc/apparmor.d# /etc/init.d/mysql2 start
* Starting MySQL database server mysqld

Networking, Uncategorized

Listing Ports and Active connections w/ Netstat

January 14th, 2010

Who’s connected to my server?

root@donkey:~# netstat -ntla | grep "80"
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN
tcp        0      0 192.168.1.5:80          206.169.197.253:20277   ESTABLISHED

Grep port for host.

root@donkey:~# lsof -i tcp:80 | grep "20277"
apache2 20114 www-data    9u  IPv4 9576072       TCP donkey.local:www->206-169-197-253.static.twtelecom.net:20277 (ESTABLISHED)

Kill Connection

kill -9 20114

Uncategorized

Vi Editing Basics

January 14th, 2010

Basic Commands

:w  - write file

:wq - write and quit

:q! - exit, ignore changes

:set number - show line numbers

:set nonumber - turn off line numbers.


Text Mode

dd - Delete lines

2dd - delete 2 lines

p  - paste buffer lines

w - skip to mext word

a - append after cursor

i - insert at cursor

o - open new line after current line

r - replace character.

yy - yank current line

Searching

/string - search forward for string

?string - search back for string.

/tel[a-z] - search for ‘tel’ with last letter a-z

Find and Replace - ‘g’ stand for global, all lines in document.

:s/OLDWORD/NEWWORD/g

Search and replace,escaping slashes.

To replace the phrase /var/lib/mysql with /home/mysql you need to escape each ‘/’ with a ‘\’. For example…

:%s/\/var\/lib\/mysql/\/home\/mysql/

Regular Expressions

/Hello/ - search for word ‘Hello’ in line.

/^Hello$/ - search and matches if Hello is only word on line.

Recording Macros

qv - start record mode for macro ‘a’. second letter should be lower case character.

q - stop record

@v - to paste recorded text

9 @v - repeat v macro 9 times.
Save Settings

You can save persistent changes for vi by editing ~/.exrc

set number
….

Uncategorized, mysql

VNC Server displays blank grey screen

December 21st, 2009

If you connect to vncserver and get a blank grey screen with a consol window then this means that the ~/.vnc/xstartup script is not initializing the windows manager correctly.

For ubuntu, edit the script (comment out lines) to the following…

#!/bin/sh
# Uncomment the following two lines for normal desktop:
# unset SESSION_MANAGER
# exec /etc/X11/xinit/xinitrc
[ -x /etc/vnc/xstartup ] && exec /etc/vnc/xstartup
[ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources
#xsetroot -solid grey
#vncconfig -iconic &
#xterm -geometry 80×24+10+10 -ls -title “$VNCDESKTOP Desktop” &
gnome-session
#twm &
\
Then, run ‘vncserver’ to create a new session based on the above script!

Uncategorized

Setup proxy w/ Squid

November 20th, 2009

Setup Squid for Linux

Internet Browser Client Setup

Firefox > preferences > settings > Manual Proxy Configuration

IP: (ip for squid server) PORT: 3128 (defautol squid port)

Squid Server Setup

apt-get install squid3
pico /etc/squid3/squid.conf

Add the following…

acl client1 src 10.10.10.2/255.255.255.0
http_access allow client1

Restart squid.

Install SARG reporting tool

sudo apt-get install sarg
edit /etc/squid/sarg.conf
modify squid log file location to:
/var/log/squid3/access.log

run ’sarg’

look in /var/www/squid-reports for html files of acces logs!

Works!

Uncategorized

Setting up a cron job in /etc/cron.hourly,cron.daily,etc…

October 28th, 2009

If you have a script that you want to setup at varying time intervals, including but not limited to certain days of the week or specific hours in a day then those commands should go in /etc/crontab.

However, if you want to simply run your script on an hourly or daily basis you can simply put a sym link in /etc/cron.daily or hourly.

Best way to do this is put your actual script in /usr/bin and create a sym link in the appropriate cron directory.

/etc/cron.daily$ sudo ln -s /usr/bin/memcache_save_data.py ./memcache_save_data

Bingo! No more to do. /etc/crontab runs all files in this folders automatically

Note: Cron does not allow you to have file extensions on files in /etc/cron dirs. Also, if you need to pass parameters to the script then you need to run this in /etc/crontab as there is no way to specify these with the above option.

Uncategorized

Mysql: Alternatives to Replication w/ Remote Dumps

October 15th, 2009

If you are unable to get the proper permissions/access on your system because of a shared hosting environment then you may just want to dump your local database and send it to a remote server. This is a simple way to setup a daily copy.

1. Create user called ‘remote’ on the slave (destination for backup) that you can access from the master (the data that needs to be backed up)

mysql> GRANT REPLICATION SLAVE, REPLICATION CLIENT ON *.* to remote@'MASTERIPADDRESS' IDENTIFIED BY 'PASSHERE';
Query OK, 0 rows affected (0.07 sec)

mysql> flush privileges;

Note:
*REPLICATION CLIENT Gives the right to the user to ask where the slaves/masters are.
*REPLICATION SLAVE Needed for the replication slaves (to read binlogs from master).

IMPORTANT! - Make sure /etc/mysql/my.cnf bind-address is not 127.0.0.1 - if so comment. If change, you will need to restart mysql.

2. Verify you can connect from the master to the slave.

# mysql -h ipaddresstoslave -u remote -p

3. Create a script to run the commands.

#!/usr/bin/python
import os
import datetime
file='mysql-dump-all-%s.sql' % (datetime.date.today())
print "Creating backup file..."
os.system('mysqldump --all-databases -u jester_root -pREMOTEPASS > ~/'+file)
print "Dumping to titan..."
os.system('mysql -h IPTOSLAVE -u remote -pREMOTEPASS < ~/'+file)
print "Deleting local sql file..."
os.system('rm ~/'+file)

4. Add file to cron

Done!

Uncategorized

Mysql Replication & Load Balancing

October 15th, 2009

What is replication and what should it is and what it isnt…

Master-Master - synchronous replication where each server updates one another.

Pros: Used for hot standby situation. Write to one master only to avoid primary key collisions.
Cons: Network intensive.

Master-Slave - Primary used to share the load. Writes go to master from client,reads go to slave.

Pros: Asyncronous updates (one way) Simple, inexpensive. Hot cache from slave.
Cons: Not good for heavy write loads.

source: http://www.mysqlperformanceblog.com/2009/11/13/finding-your-mysql-high-availability-solution-%E2%80%93-replication/

http://dev.mysql.com/doc/refman/5.1/en/replication.html

How to setup mysql replication

Create new replication user on master.

mysql> GRANT REPLICATION SLAVE, REPLICATION CLIENT ON *.* to slave@'SLAVEIPADDRESS' IDENTIFIED BY 'PASSHERE';
Query OK, 0 rows affected (0.07 sec)

IMPORTANT!!! - make sure /etc/mysql/my.cnf 127.0.0.1 for bind-address is commented out or cannot connect from the slave!


#bind-address = 127.0.0.1

Now verify can connect from slave:


root@slave:/var/log# mysql -h MASTERIP -P 3306 -u slave -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 60
Server version: 5.1.37-1ubuntu5-log (Ubuntu)
Type 'help;' or '\h' for help. Type '\c' to clear the buffer.
mysql>

Configure master.

edit /etc/my.cnf

Add the following.

[mysqld]

log-bin=mysql-bin
server-id=1 #master id

Make backup of Master.


root@donkey [~]# mysqldump –all-databases -u root -p > mysql-dump-master.sql
Enter password:

Write down the position of the master status for later…

mysql> show master status;
+------------------+----------+--------------+------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+------------------+----------+--------------+------------------+
| mysql-bin.000001 | 106 | | |
+------------------+----------+--------------+------------------+
1 row in set (0.00 sec)

Shudown mysql


root@donkey [~]# sudo /etc/init.d/mysql stop Shutting down MySQL.

Transfter file to Slave.

scp mysql-dump-all-10-15-09.sql root@slave:~

Import Data

root@titan:~# mysql -u root -p < ~/mysql-dump-all-10-15-09.sql
Enter password

Slave /etc/mysql/my.cnf

[mysqld]
server-id = 2
log_bin = mysql-bin.log

Now set master from slave.

mysql> CHANGE MASTER TO MASTER_HOST='205.234.235.89',MASTER_USER='replication_user', MASTER_PASSWORD='PASSHERE',MASTER_LOG_POS=106,MASTER_LOG_FILE='mysql-bin.000001';
Query OK, 0 rows affected (0.00 sec)

*log file and position should be the output from master ’show master status’; command.
*in need to reset the above then do ’stop slave’, ‘reset slave’ and issue the CHANGE MASTER TO cmd again.

mysql> start slave
-> ;
Query OK, 0 rows affected (0.00 sec)

Verify Master/Slave are replicating. ‘Slave IO Running’ and ‘Slave SQL Running’ should both be set to “YES”

mysql> show slave status\G;
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 10.139.68.174
Master_User: slave
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000002
Read_Master_Log_Pos: 43981
Relay_Log_File: mysqld-relay-bin.000005
Relay_Log_Pos: 33858
Relay_Master_Log_File: mysql-bin.000002
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 43981
Relay_Log_Space: 33858
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 0
1 row in set (0.00 sec)

Check master to see if positon of slave ans master are in sync!

Done!

Uncategorized

Setting up a new mysql user.

October 9th, 2009

#Grant options: ALL, SELECT, UPDATE, INSERT
mysql> GRANT SELECT ON *.* to 'someuser'@'IPCONNECTINGFROM or %' IDENTIFIED BY 'passwordhere';
Query OK, 0 rows affected (0.00 sec)

mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)

Verify setup…

mysql> select host, password, user from user;

Uncategorized

Install Mod Python for Apache!

September 16th, 2009

Install module

sudo apt-get install libapache2-mod-python

Add python directives to /etc/apache2/sites-enabled/000-default

<Directory /var/www/>
Options Indexes FollowSymLinks MultiViews
AllowOverride None
Order allow,deny
allow from all
AddHandler mod_python .py
PythonHandler mod_python.publisher
PythonDebug On
</Directory>

Restart apache.

sudo /etc/init.d/apache2 restart

Create test page:

def index(req):
return "donkey";

Bingo!

Uncategorized

Installing RPM on Ubuntu

September 10th, 2009

Need to convert RPM to DEB

apt-get install alien.

jesterj@1029384756:~$ sudo alien -k ./Desktop/VMware-Player-2.5.3-185404.i386.rpm
Warning: Skipping conversion of scripts in package VMware-Player: postinst prerm
Warning: Use the –scripts parameter to include the scripts.
vmware-player_2.5.3-185404_i386.deb generated
jesterj@1029384756:~$ sudo dpkg -i vmware-player_2.5.3-185404_i386.deb
Selecting previously deselected package vmware-player.
(Reading database … 194000 files and directories currently installed.)
Unpacking vmware-player (from vmware-player_2.5.3-185404_i386.deb) …
Setting up vmware-player (2.5.3-185404) …
jesterj@1029384756:~$

Uncategorized

PHP: Save As Dialog box for file download.

August 25th, 2009

< ? PHP
header("Content-type: text/plain");
header("Content-Disposition: attachment; filename='temp.txt'");
readfile("./temp.txt");
?>

Uncategorized

Python: Convert Epoch to GM Time.

August 24th, 2009

>>> import time
>>> timestamp=float(’1251157271.796′)
>>> print time.gmtime(timestamp)
time.struct_time(tm_year=2009, tm_mon=8, tm_mday=24, tm_hour=23, tm_min=41, tm_sec=11, tm_wday=0, tm_yday=236, tm_isdst=0)
>>> start_time=time.gmtime(timestamp)
>>> for x in start_time:
…     print x

2009
8
24
23
41
11
0
236
0

Uncategorized

Common PHP Memcache Error - enable the module!

August 24th, 2009

memcache class  not found in php page.

problem: i’ve install memcache for php, restarted apache but get the follwoing
error…
“Fatal error: Class ‘Memcache’ not found in /var/www/loadscript/index.php on line 2″

resolution: Add this line to php.ini and restart apache

extension=memcache.so

Uncategorized

Storing data in Memcache with Python

August 24th, 2009

$ apt-get install memcached python-memcache

jesterj@jesterj-laptop:~$ python
Python 2.6.4 (r264:75706, Dec 7 2009, 18:45:15)
[GCC 4.4.1] on linux2
Type “help”, “copyright”, “credits” or “license” for more information.
>>>import memcache
>>>memcache = memcache.Client(['10.10.10.45:11211'])
>>>datavar=”some data here”
>>>memcache.set(’1234′,datavar,60)
>>>memcache.get(’1234′)

’some data here’

Ref: http://dev.mysql.com/doc/refman/5.1/en/ha-memcached-interfaces-python.html

Do the same above to connect from remote machine…

Uncategorized

Sub-process /usr/bin/dpkg returned an error code (1)

August 14th, 2009

Error uninstalling/installing Ubuntu package.

If you have a problem with a certain package run this…

sudo dpkg -r <packagename>

Then reinstall with apt!

Uncategorized

Python: Tuples vs Lists

August 13th, 2009

Tuple are immutable. Lists are mutable.

a=(1,2,3) #tuples have ()
a[0]=100

Error! Tuple values cannot be changed!

a=[1,2,3] #list have brackets
a[0]=100

success!

Note: dicts have curly braces ie: a={}

Uncategorized

Create Raid0+1 Array w/ Mdadm (Striping & Mirroring)

July 9th, 2009

Install mdadm
apt-get install mdadm

Create the fisrt striped set:

sudo mdadm –create –force /dev/md0 –level=stripe –chunk=4096 –raid-devices=2 /dev/sdc /dev/sdd

Create the second striped set:
sudo mdadm –create –force /dev/md1 –level=stripe –chunk=4096 –raid-devices=2 /dev/sde /dev/sdf

You now have two 2tb striped arrays. We now want to take those and tell mdadm to mirror them w/ Raid 1, hence the
name Raid0+1.

Raid 1 both of your stripe sets

sudo mdadm –create –force /dev/md2 –level=mirror –chunk=4096 –raid-devices=2 /dev/md0 /dev/md1

Partition the drive w/ fdisk

fdisk /dev/md2
-Create a new partition.
-Write to disk

Next, format the new drive

mkfs.ext3 /dev/md2

Finally, mount the drive and use!

mount -t ext3 /dev/md2 /mnt/raid1

View the details of the array…

mdadm -D /dev/md2

$ cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4] [raid0] [raid1]
md2 : active raid1 md1[1] md0[0]
1953521600 blocks [2/2] [UU]
[===========>.........] resync = 59.3% (1160324480/1953521600) finish=973.8min speed=13574K/sec

md1 : active raid0 sdf[1] sde[0]
1953521664 blocks 4096k chunks

md0 : active raid0 sdd[1] sdc[0]
1953521664 blocks 4096k chunks

unused devices:

Uncategorized

Record all traffice on port 80

June 29th, 2009

sudo tcpdump -n -i eth0 -s 0 -w output.txt src or dst port 80

Networking, Uncategorized

Encrypt web traffic

June 29th, 2009

-Firefox > Preferences > Manual Proxy configuration > Socks Host: localhost Port: 9999, socksv5.

-Next open linux terminal and setup up tunneling.

$ssh -CND 9999 user@remoterserver.com

*Leave terminal window open.

To verify web traffic is coming from the host go to www.whatismyip.com and the server ip should be listed from where data is being tunnelled through!

Uncategorized

Python: Unions of lists and sets.

June 19th, 2009
a=set(['apple','apple','orange','apple','pear','berry'])
b=set(['pear','coconut','strawberry'])
a | b

set(['strawberry', 'coconut', 'apple', 'orange', 'pear', 'berry'])

a.union(b)

set(['strawberry', 'coconut', 'apple', 'orange', 'pear', 'berry'])

Uncategorized

Python: Reading and Writing to a Word (.doc) file

June 19th, 2009

Need to install ‘antiword’ debian package then read output from CLI.

import commands
output=commands.getoutput('antiword /home/jesterj/Desktop/test.doc')
print output

Mary had a little lamb. Mary had a little lamb. Mary had a little lamb.
Mary had a little lamb. Mary had a little lamb. Mary had a little lamb.
Mary had a little lamb. Mary had a little lamb. Mary had a little lamb.
Mary had a little lamb. Mary had a little lamb. Mary had a little lamb.
Mary had a little lamb. Mary had a little lamb. Mary had a little lamb.
Mary had a little lamb. Mary had a little lamb. Mary had a little lamb.
Mary had a little lamb. Mary had a little lamb. Mary had a little lamb.
Mary had a little lamb. Mary had a little lamb. Mary had a little lamb.
Mary had a little lamb. Mary had a little lamb. Mary had a little lamb.
Mary had a little lamb. Mary had a little lamb. Mary had a little lamb.

Write to word…

Pretty easy here. Just write to a file and append with .doc.

f=open("newfile.doc","wr")
f.write(output)
f.close()

Badow!

Uncategorized

Python: Reading & Writing to Excel

June 19th, 2009

install ‘xlrd’ for reading…
Download: http://www.lexicon.net/sjmachin/xlrd.htm

root@donkey:~# sudo apt-cache search xlrd
python-xlrd - extract data from MS Excel spreadsheet files

import xlrd
wb=xlrd.open_workbook('/home/jesterj/Desktop/JJ-MonthlyNumbersNew-6.18.09.xls')
wb.sheet_names()

[u'Sheet1', u'Sheet2', u'Sheet3']

sh=wb.sheet_by_index(0)
for rownum in range(sh.nrows):
    print sh.row_values(rownum)

Ref: http://scienceoss.com/read-excel-files-from-python/

Install ‘xlwt’ for writing excel files…
Download: http://pypi.python.org/pypi/xlwt

import xlwt
from datetime import datetime

font0 = xlwt.Font()
font0.name = 'Times New Roman'
font0.colour_index = 2
font0.bold = True

style0 = xlwt.XFStyle()
style0.font = font0

style1 = xlwt.XFStyle()
style1.num_format_str = 'D-MMM-YY'

wb = xlwt.Workbook()
ws = wb.add_sheet('A Test Sheet')

ws.write(0, 0, 'Test', style0)
ws.write(1, 0, datetime.now(), style1)
ws.write(2, 0, 1)
ws.write(2, 1, 1)
ws.write(2, 2, xlwt.Formula("A3+B3"))

wb.save('example.xls')

Uncategorized

Computing a Symmetrical Difference in sets and lists.

June 18th, 2009

Symmetrical difference calculates the difference of not whats in the just one list. It calculates the items that are not common in both lists!

list1=['donkey','balls']
list2=['donkey','scrotom']
set1=set(list1)
set2=set(list2)
print set1

set(['donkey', 'balls'])

print set2

set(['donkey', 'scrotom'])

symdiff = set1 ^ set2
symdiff

set(['balls', 'scrotom'])

Uncategorized

Python argv & getopts

June 10th, 2009
def main():
cases=['auth','transact','dns','network','ssl']
print sys.argv
try:
if sys.argv[1]==’-t’:
if sys.argv[2] not in cases or len(sys.argv[2])==0:
print “*Must specify a valid case: %s ” % cases
exit()
else:
case=sys.argv[2]

if case==’auth’:
test_authentication()
if case==’transact’:
test_transaction_problem(transact_url)
if case==’dns’:
test_dns(hosts)
if case==’network’:
test_network(hosts)
if case==’ssl’:
test_ssl(hosts)
else:
print “You must specify a valid test %s” % cases
except:
print “You must specify option -t followed by test option %s” % cases

Uncategorized

Create emtpy file with specified size for testing.

June 9th, 2009

dd if=/dev/zero of=file_to-create bs=1k count=4000000

*the above will create a 4gb file called ‘file_to-create”

Uncategorized

Python: Get http status code

June 8th, 2009

Using urllib…

#!/usr/bin/python
import urllib
c=urllib.urlopen('http://nike.com')
print c.getcode()

200

or try with httplib…

#!/usr/bin/python
import httplib
conn = httplib.HTTPConnection("www.python.org")
conn.request("GET", "/index.html")
r1 = conn.getresponse()
print r1.status

200

Networking, Uncategorized

Count the occurence of an item in a list. python

June 2nd, 2009
#!/usr/bin/python

items=['klm','klm','pia','nwa','klm','al','al','al','ri','pia','klm']

#get unique items from list
get_unique=[]
for a in items:
if a not in get_unique:
get_unique.append(a)

#dict for storage of key,occurence
dict={}

for x in items:
for y in get_unique:
if x in y:
if x in dict:
val=dict[x]
val+=1
dict[x]=val
else:
dict[x]=1

print dict

</code>

Uncategorized

Python: Method order

May 29th, 2009

*NOTE: def() method/function must be defined before it is called. If the def is below where it is called will get a ‘NameError: name ‘donkey’ is not defined’

def donkey():

print “hello”

a=donkey()

Uncategorized

Python: Return a list or dictionary from a method

May 29th, 2009
#!/usr/bin/python
def mylist(a):
fruit=[]
fruit.append(a)
return fruit

b=mylist(”donkey”)
print b

Uncategorized

Get zipfile contents info. python.

May 28th, 2009
import zipfile

a=zipfile.ZipFile('/path/to/zipfile.ZIP')
for x in a.namelist():
...     if x.endswith('.zip'):
...             info=a.getinfo(x)
...             print "%s %s" % (info.filename, info.file_size)
...

123.zip 4563
254.zip  5435
donkey.zip 8979

Uncategorized

Find occurence of items in list.

May 27th, 2009

letterDict = {}
all_letters_list =
['a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r'
,'s','t','u','v','w','x','y','z']
test_string = “in mississippimississippimississippimississippi pigs can
fly like the wind”

# saving the dict
for letter in test_string:
letterDict[letter] = letterDict.get(letter, 0) + 1

print letterDict

=========

print ‘-’ * 60
print ‘Examine the line below to see how it worked ‘
print ‘ 1 - inserting:  letterDict[letter] = letterDict.get(letter, 0) +
1  - self initiates with value of 0 and increments +1′
print ‘ 2 -  reading :  letterDict.get(testLetter, 0)  - when reading -
insert a default count of 0 for those letters not found.’
print ‘-’ * 60
print ‘test_string = ‘, test_string
print
print ‘letterDict = ‘, letterDict
print
print ‘Histogram (count) of letters in test string’

# reading the dict, but inserting default 0 for letters not found.
for testLetter in all_letters_list:
print ‘%s - %2s’ % (testLetter, letterDict.get(testLetter, 0))

Uncategorized

Python: Get similar matching strings

May 22nd, 2009
import difflib
a=difflib.get_close_matches
a("Phil Collins", ["Phil Colins","PhilCollins","Phil Collins","Phil Collins - Genesis"])

['Phil Collins', 'PhilCollins', 'Phil Colins']

Uncategorized

Delete all empty directories in current path.

May 21st, 2009

find -depth -type d -empty -exec rmdir {} \;

Uncategorized

Resolving Drive that won’t unmount.

May 20th, 2009

# umount /dev/sdb1
umount: /dev/sdb1: device is busy
umount: /dev/sdb1: device is busy
# fuser -m /dev/sdb1
/dev/sdb1: 3322

(3322 is id of the process that uses the device. if you like to know what it is, you may try this:

# ps aux | grep 3322)

# kill -9 3322
# umount /dev/sdb1
#

Uncategorized

Creating and calling a class - python

May 14th, 2009
Type "help", "copyright", "credits" or "license" for more information.
import Cfg #class you just created
c = Cfg.Cfg()
c.test()
donkey

from Cfg import Cfg
c = Cfg()
c.test
bound method Cfg.test of <Cfg.Cfg instance at 0xb7ce35ec>>
c.test()
donkey

Uncategorized

Grab last element of a list/string

May 5th, 2009

>>> path=’/donkey/file.cfg’
>>> var=path.split(’/')
>>> print var
['', 'donkey', 'file.cfg']
>>> var[-1]
‘file.cfg’

or can do

>>>varv.pop()
‘file.cfg’

#note you must assign path.split(’/') to a string or path.string will
just print out the last letter of the 3rd element which is ‘g’

Uncategorized

Removing garbage data from string

May 4th, 2009

Many times when you parse a file using split() or the csv module you return strings that have garbage characters in the line. Take the below string for example…

>>> badstring=’\xef\xbb\xbf#Some information’
>>> print badstring
#Some information
>>> if badstring.startswith(’#'):
…     print “Hello”

>>>

The above doesn’t print ‘Hello’ becuase of the garbage data. So we need to create a method to remove the junk chars.

Here is what we do….

def remove_garbage():

Uncategorized

Count number of items in list.

April 30th, 2009
items=['a','b','c']
size=len(items)

print size

# 3

Uncategorized

Check for open ports/services on remote machine

April 20th, 2009

$ telnet titan.zipcon.net 22
Trying 63.249.22.50…
Connected to titan.zipcon.net.
Escape character is ‘^]’.
SSH-2.0-OpenSSH_5.1p1 Debian-3ubuntu1

Uncategorized

List who is logged in to your linux box

April 20th, 2009

w
14:16:50 up 2 days, 23:54,  4 users,  load average: 0.24, 0.16, 0.07
USER     TTY      FROM              LOGIN@   IDLE   JCPU   PCPU WHAT
jesterj  tty7     :0               Fri14    3days 24:30m  0.30s x-session-manag
jesterj  pts/0    :0.0             14:13    0.00s  0.10s 19.94s gnome-terminal
jesterj  pts/2    :0.0             11:52    2:23   0.20s  0.10s ssh jesterj@pse
jesterj  pts/1    :0.0             11:23    2:34   0.16s  0.06s ssh jesterj@pse

Uncategorized

View all open ports with Netstat

April 20th, 2009

$ netstat -nta
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 0.0.0.0:6024            0.0.0.0:*               LISTEN
tcp        0      0 127.0.0.1:3306          0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN
tcp        0      0 127.0.0.1:631           0.0.0.0:*               LISTEN
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN
tcp        0      0 10.139.68.153:42651     10.139.68.39:22         ESTABLISHED
tcp        0      0 10.139.68.153:51346     64.4.34.126:1863        ESTABLISHED
tcp        0      0 10.139.68.153:44434     206.255.61.214:4228     ESTABLISHED
tcp        0      0 10.139.68.153:38705     205.234.235.90:80       ESTABLISHED
tcp        0      0 10.139.68.153:38707     205.234.235.90:80       ESTABLISHED
tcp        0      0 10.139.68.153:40101     64.4.37.39:1863         TIME_WAIT
tcp        0      0 10.139.68.153:38708     205.234.235.90:80       ESTABLISHED
tcp        0      0 10.139.68.153:52217     10.139.10.45:22         ESTABLISHED
tcp        0      0 10.139.68.153:38702     205.234.235.90:80       ESTABLISHED
tcp        0      0 10.139.68.153:38709     205.234.235.90:80       ESTABLISHED
tcp        0      0 10.139.68.153:38710     205.234.235.90:80       ESTABLISHED
tcp6       0      0 :::80                   :::*                    LISTEN
tcp6       0      0 :::22                   :::*                    LISTEN

Uncategorized

Software raid voume won’t start after reboot.

March 18th, 2009

ubuntu@ubuntu:/mnt$ sudo mount -t ext3 /dev/md0 /mnt/md0

mount: wrong fs type, bad option, bad superblock on /dev/md0,
missing codepage or other error
(could this be the IDE device where you in fact use
ide-scsi so that sr0 or sda or so is needed?)
In some cases useful info is found in syslog - try
dmesg | tail  or so


$ sudo mdadm –assemble /dev/md0 /dev/sdd /dev/sdb /dev/sdc
mdadm: /dev/md0 has been started with 3 drives.

$sudo mount /dev/md0 /data/

OR….

Simply run this command.

mdadm --detail --scan >> /etc/mdadm/mdadm.conf

Mount again and all should work!

Uncategorized

Python: Get hostname

March 11th, 2009
import sys, socket

hostname = socket.gethostname()
print "Host name:", hostname

Uncategorized

Create ISO in Linux

March 9th, 2009

To create an image (link to a more verbose explanation) use dd on an unmounted CD/DVD drive:
dd if=/dev/cd of=cd.iso

Uncategorized

Create Linux Command Aliases

March 5th, 2009

add command to the bottom of the .bashrc file

alias cvsu='cvs -q update -P -d'

$source ~/.bashrc

now run the new command
$ cvsu

Uncategorized

Accessing MySQL Remotely

February 25th, 2009

Say for example, you have a web page you want to access some data on but the database is located on a different server. To do this you will need to add another user to the mysql.users table and specify the remote location where you are accesing from.

$ mysql -u root -p

mysql>  GRANT ALL PRIVILEGES ON database_name.* TO myUser@host_name IDENTIFIED BY 'pass' WITH GRANT OPTION;

*user ‘%’ for hostname if wanting to accept connections from all hosts.

mysql> Flush priviledges

Next, edit your /etc/mysql/my.cnf file:

Comment the bind-address

#bind-address: 127.0.0.1  //if enabled with disallow connections from a remote host.

Restart mysql:

# /etc/init.d/mysql restart

Uncategorized, mysql

Python Search and Replace Script

January 30th, 2009

Usage: python ./scriptname.py oldword newword sourcfile.txt newfile.txt

#!/usr/local/bin/python
import os, sys
usage = "usage: %s search_text replace_text [infile [outfile]]” %         os.path.basename(sys.argv[0])

if len(sys.argv) < 3:
     print usage
else:
stext = sys.argv[1]
rtext = sys.argv[2]
input = sys.stdin
output = sys.stdout
if len(sys.argv) > 3:
input = open(sys.argv[3])
if len(sys.argv) > 4:
output = open(sys.argv[4], ‘w’)
for s in input.xreadlines():
output.write(s.replace(stext, rtext))

Uncategorized

Test your network speed w/ Iperf

January 22nd, 2009

Install iperf

apt-get install iperf

Listen on machine #1

$ iperf -s -P 2 -i 5 -p 5999 -f M

Send packets on machine #2

iperf -c backupserv -P 1 -i 5 -p 5999 -f M -t 60 -T 1

Results:
————————————————————
Client connecting to backupserv, TCP port 5999
TCP window size: 0.02 MByte (default)
————————————————————
[  3] local 192.168.1.5 port 49711 connected with 192.168.1.3 port 5999
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0- 5.0 sec    562 MBytes    112 MBytes/sec
[  3]  5.0-10.0 sec    562 MBytes    112 MBytes/sec
[  3] 10.0-15.0 sec    561 MBytes    112 MBytes/sec
[  3] 15.0-20.0 sec    561 MBytes    112 MBytes/sec
[  3] 20.0-25.0 sec    562 MBytes    112 MBytes/sec
[  3] 25.0-30.0 sec    561 MBytes    112 MBytes/sec
[  3] 30.0-35.0 sec    561 MBytes    112 MBytes/sec
[  3] 35.0-40.0 sec    561 MBytes    112 MBytes/sec
[  3] 40.0-45.0 sec    561 MBytes    112 MBytes/sec
[  3] 45.0-50.0 sec    561 MBytes    112 MBytes/sec
^C[  3]  0.0-50.8 sec  5707 MBytes    112 MBytes/sec

Linux, Networking, Uncategorized

Create virtual network interface

January 22nd, 2009

sudo /sbin/ifconfig eth0:0 192.168.1.50 up

sudo /sbin/ifconfig eth0:0 192.168.1.50 down

Uncategorized

Create an empty 500mb file.

January 22nd, 2009

dd if=/dev/zero bs=1M count=150 | bzip2 -c > 150mb.bz2

Linux, Uncategorized

Rsync python backup script via cron

January 20th, 2009

Run script in crontab for server backups, cuts down on number of lines in crontab and staggers rsync over time.

01 * * * * /path/to/file/backu_script.py

#!/usr/bin/python
import time
import os
current_hour=time.strftime("%H")
backup_user="pse_backup"

#element 0 = servername, element 1 = hour to backup
backup_time={'pse01':'10','pse02':'17'}
servers={'pse01':['/home','/var/log'], ‘pse02′:['/home','/var/log','/etc']}

for k, v in backup_time.iteritems():
hour = v
server = k
if hour == current_hour:
print server + ” is being backed up!”
for name, paths in servers.iteritems():
if server == name:
#Run Rsync cmd.
for path in paths:
os.system(”rsync -van -e ’ssh -i /home/pse_backup/.ssh/pse_backup_key’ “+backup_user+”@”+server+”:” + path + ” /data/backups/” + server + path)

Uncategorized

Combine Multiple PDF’s into one file

January 16th, 2009

gs -dBATCH -dNOPAUSE -q -sDEVICE=pdfwrite -sOutputFile=finished.pdf file1.pdf file2.pdf

Uncategorized

Create Apache2 self sign cert

December 29th, 2008

$ openssl genrsa -des3 -out server.key 1024

$ openssl rsa -in server.key -out server.key.insecure

$ openssl req -new -key server.key -out server.csr

$ openssl req -noout -text -in server.csr

$ sudo openssl x509 -req -days 3650 -in server.csr -signkey server.key -out server.crt

$ sudo mv server.crt /etc/ssl/certs
$ sudo mv server.key.insecure /etc/ssl/private/server.key
$ sudo mv server.key /etc/ssl/private/server.key.secure

$ openssl s_client -connect pse02:443

Add :443 and SSL cert info to file

$ cat /etc/apache2/sites-available/default
NameVirtualHost *:443

ServerName server.name
ServerAdmin email@host.com

SSLEngine On
SSLCertificateFile /etc/ssl/certs/server.crt
SSLCertificateKeyFile /etc/ssl/private/server.key

Apache, Uncategorized

Getting Airport Express working with Xbox 360

December 13th, 2008

Problem: I have an airport express in my basement but my xbox is located upstairs. I want to avoid running wire upstairs as well as purchasing the $80 wireless adapter for the xbox. Since i have a spare airport express i want to set this up in brigding mode. It does work but there are quite a few steps. See below:

How to configure airport xpress with Xbox

  1. Setup main base station to work correctly
  2. Set channel to ‘1′ or the same on both.
  3. Get the airport id from each wireless unit (not the WAN id) - you will need this later.
  4. On the airport for the xbox make sure you have these settings;
  5. Wireless Mode: Create Wiresless Network (home router)
  6. Connect using: Airport (WDS)
  7. Mac Address: (enter the mac for the main base station)
  8. Configure: Using DHCP
  9. Under WDS Table
  10. enable this base sataion as ‘remote base stations’
  11. Enter airport ID for main base station.
  12. On the main base station go to the WDS tab
  13. enable as main base station
  14. add mac address of xbox airport express

Update both and restart!

Networking, Uncategorized

Insert multiple records in MySQL with one Stmt.

December 11th, 2008
INSERT INTO table (id, name) values ('1','test');
INSERT INTO table (id, name) values ('2','test2');
INSERT INTO table (id, name) values ('3','test3');

However, it is much quicker to do it this way:

INSERT INTO table (id, name) values ('1', 'test1'), ('2', 'test2'), ('3', 'test3');

Much faster!

Uncategorized, mysql

Get IP Address in Python on Linux

December 5th, 2008
#!/usr/bin/python

import socket
import fcntl
import struct

def get_ip_address(ifname):
    s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    return socket.inet_ntoa(fcntl.ioctl(
        s.fileno(),
        0x8915,  # SIOCGIFADDR
        struct.pack('256s', ifname[:15])
    )[20:24])

print get_ip_address(’lo’)
print get_ip_address(’eth0′)

Uncategorized

Rsync - How to backup your server the right way!

December 4th, 2008

Describe How to Set Up Rsync here.install rsync on both client and server. they should have the same versions of rsync installed.

$apt-get install rsync

Create new ssh key w/out password

$ssh-keygen -f rsync_key -C 'Rsync Key' -N '' -t rsa -q

copy key to server

ssh-copy-id -i /home/user/.ssh/rsync_key.pub jesterj@server
jesterj@server's password:
Now try logging into the machine, with "ssh 'jesterj@server'", and check in:

  .ssh/authorized_keys

test login with specified key

slogin -i ~/.ssh/rsync_key backupuser@server1

Make sure rsync is running.

/etc/init.d/rsync start

Test from local machine to verify you can connect:

$ rsync -e ssh localhost::

Now run this command from backup server to sync files. the ‘n’ aoption does a test run. remove it to do an actual sync.

rsync -van --delete --ignore-errors -e "ssh -i /home/user/.ssh/rsync_key" /source/path jesterj@server:/backup/path
Add command to cron to repeat.

*The '--delete' means to delete items that don't exist in the backup location, --ignore-errors, continues the  backup
even though errors exist.

Uncategorized

Software RAID Setup under Ubuntu

December 3rd, 2008

install ‘mdadm’. to reconfig run ‘dpkg-reconfigure mdadm’

CREATE THE ARRAY

raid5: mdadm - -create - -force /dev/md0 - -level=raid5 - -chunk=64 - -parity=left-symmetric - -raid-devices=3 /dev/sdb /dev/sdc /dev/sdd

raiad0/Striping:  mdadm - -create /dev/md0 - -level=stripe - -chunk=4096 - -raid-devices=2 /dev/sda /dev/sdb

STOP MDADM
sudo mdadm –stop /dev/md0 (if resource busy)

CREATE PARTITION
sudo fdisk /dev/md0

FORMAT
sudo mkfs.ext3 /dev/md0

MOUNT DRIVE
mount /dev/md0 /mnt/raid

VIEW RAID DETAILS

mdadm -D /dev/md0

-add to /etc/fstab for mount on boot!
-do a ‘df’ to view your new system!

SAVE CONFIG FOR REBOOT

mdadm --detail --scan >> /etc/mdadm/mdadm.conf

Uncategorized

Python: Dictionary examples.

December 2nd, 2008
#!/usr/bin/python
#dictionary
source={}

source["123"] = “filename.txt”
source["444"] = “fit”

for i,v in source.iteritems():
print “%s %s” % (i,v)

# python dict.py
123 filename.txt
444 fit

Uncategorized

Using the ‘Find’ command.

November 24th, 2008

To find files (starting in the current directory) with names ending with .data and to print their names, try this:

find . -name ‘*.data’ -print
company.data
donor.data
grades.data
sorted.data
words.data

To find files larger than 40K and print the file names and details (use a minus sign instead of a plus sign to find files smaller than a certain size), issue this command:

find . -size +40k -ls
-rw-rw-r– hermie users 56720 Jan 16 12:42 bigfile
-rw-rw-r– hermie users 415206 Feb 27 21:37 largefile
-rw-rw-r– hermie users 315428 Jan 07 05:23 hugefile

To find files ending with .dat that are smaller than 100K, enter

find . -name *.txt -size -100k -ls
-rw-rw-r– hermie users 26720 Feb 06 23:52 recipes.txt
-rw-rw-r– hermie users 506 Feb 18 18:45 poem.txt

To find files that have not been accessed for over 30 days and delete them (by sending their names to the rm command), enter

find . -atime +30 -exec rm {} \;

To find directories (starting in the junk directory) and conditionally delete them (by sending their names to the rmdir command), enter

find junk -type d -ok rmdir {} \;

From: http://lowfatlinux.com/linux-find.html

Uncategorized

Annoying PC Beep in Linux

November 6th, 2008

Problem: Every time you mistype something or  the cmd prompt doesn;t understand your input I get a annoy system beep that annoys me and my co-workers. I don’t want to disable sound all together as I would like to still listen to music.
Solution: Blacklist that sucker!

In /etc/modprobe.d/blacklist add the following

“blacklist pcspkr”

Then run ’sudo modprobe pcspkr’

*If that doesn’t work you can remove it permanently by issueing: ’sudo modprobe -r pcspkr’

Now no more systems beeps!

Ref: http://ubuntu-tutorials.com/2007/07/26/turning-off-the-system-hardware-beep-linux-tutorial/

Uncategorized

Setting up cron jobs

November 4th, 2008

Need to setup a script to run at a specified time every day, week or month. Use cron!

While logged into linux you can view your crontab file by typing the following

$crontab -l

# m h  dom mon dow   command
35 15 * * *    python /home/jesterj/fogbugz_sql_backup.py

Notice the time parameters followed by the system command. The format is follows…

#min | hour  | day month | month | day of week
10 12 * * *  <command here>

The above script would execute at 12:10pm every day.

To edit this file issue the command with an -e option

$crontab -e

To run a command with sudo priviledges, simply issue a ’sudo’ before the cron command like so.

$sudo crontab -e

That’s it!

Uncategorized , ,

Upcoming topics…

October 31st, 2008

-sample python scripts

-nagios configure and install

-saving ssh keys.

-virtual box setup (bridging network)

Uncategorized

grep the file system

October 31st, 2008

Find all files matching a particular phrase on your file system.

#the minus R option searches recurisively through files.

~$ sudo grep -R “shutdown” /var/log/messages

Oct 28 09:25:00 jeremiah-laptop nagios2: Successfully shutdown… (PID=6230) Oct 29 16:03:33 jeremiah-laptop nagios2: Successfully shutdown… (PID=6206)

Uncategorized