Saturday, October 19, 2013

Step-by-Step Bugzilla Installation Guide for Linux

http://www.thegeekstuff.com/2010/05/install-bugzilla-on-linux/

Bugzilla is the best open source bug tracking system. Very simple to use with lot of features. Bugzilla allows you to track the bugs and collaborate with developers and other teams in your organization effectively.
This is a detailed step-by-step bugzilla installation guide for Linux.

1. Verify Perl Version

Make sure your perl version is >= 5.8.1 as shown below.
# perl -v

This is perl, v5.8.8 built for i386-linux-thread-multi
Most Linux distributions comes with perl. If you don’t have it on yours, download and install it from corresponding distribution website.

2. Install MySQL Database

Make sure your MySQL version is >= 4.1.2 as shown below.
# mysql -V
mysql  Ver 14.12 Distrib 5.0.77, for redhat-linux-gnu (i686) using readline 5.1
If you don’t have mysql, install it as using yum groupinstall, or based on LAMP install article, or based on mysql rpm article.

3. Install Apache

If you already have apache installed, make sure you are able to access it by using http://{your-ip-address}.
If you don’t have apache, install is using yum based on LAMP install article, or install apache from source.

4. Download latest Bugzilla tar ball

Download the latest stable release from bugzilla download page. Extract the bugzilla*.tar.gz file to the apache document root directory as shown below.
# cd ~

# wget http://ftp.mozilla.org/pub/mozilla.org/webtools/bugzilla-3.6.tar.gz

# cd /var/www/html

# tar xvfz /usr/save/bugzilla-3.4.6.tar.gz

5. Execute the bugzilla checksetup.pl

Bugzilla checksetup.pl program will verify whether all the required perl modules are installed. This will also display a list of all missing bugzilla modules that needs to be installed.
You can run the checksetup.pl program as many times as you like until you’ve verified all the required perl modules are installed.
Following is the output of 1st run of checksetup.pl, where is has listed all the missing optional and required modules.
# cd /var/www/html/bugzilla-3.4.6

# ./checksetup.pl --check-modules

COMMANDS TO INSTALL OPTIONAL MODULES:

             GD: /usr/bin/perl install-module.pl GD
          Chart: /usr/bin/perl install-module.pl Chart::Base
    Template-GD: /usr/bin/perl install-module.pl Template::Plugin::GD::Image
     GDTextUtil: /usr/bin/perl install-module.pl GD::Text
        GDGraph: /usr/bin/perl install-module.pl GD::Graph
       XML-Twig: /usr/bin/perl install-module.pl XML::Twig
     MIME-tools: /usr/bin/perl install-module.pl MIME::Parser
    libwww-perl: /usr/bin/perl install-module.pl LWP::UserAgent
    PatchReader: /usr/bin/perl install-module.pl PatchReader
     PerlMagick: /usr/bin/perl install-module.pl Image::Magick
      perl-ldap: /usr/bin/perl install-module.pl Net::LDAP
    Authen-SASL: /usr/bin/perl install-module.pl Authen::SASL
     RadiusPerl: /usr/bin/perl install-module.pl Authen::Radius
      SOAP-Lite: /usr/bin/perl install-module.pl SOAP::Lite
    HTML-Parser: /usr/bin/perl install-module.pl HTML::Parser
  HTML-Scrubber: /usr/bin/perl install-module.pl HTML::Scrubber
Email-MIME-Attachment-Stripper: /usr/bin/perl install-module.pl Email::MIME::Attachment::Stripper
    Email-Reply: /usr/bin/perl install-module.pl Email::Reply
    TheSchwartz: /usr/bin/perl install-module.pl TheSchwartz
 Daemon-Generic: /usr/bin/perl install-module.pl Daemon::Generic
       mod_perl: /usr/bin/perl install-module.pl mod_perl2

YOU MUST RUN ONE OF THE FOLLOWING COMMANDS (depending on which database you use):

PostgreSQL: /usr/bin/perl install-module.pl DBD::Pg
     MySQL: /usr/bin/perl install-module.pl DBD::mysql
    Oracle: /usr/bin/perl install-module.pl DBD::Oracle

COMMANDS TO INSTALL REQUIRED MODULES (You *must* run all these commands and then re-run checksetup.pl):

    /usr/bin/perl install-module.pl CGI
    /usr/bin/perl install-module.pl Digest::SHA
    /usr/bin/perl install-module.pl Date::Format
    /usr/bin/perl install-module.pl DateTime
    /usr/bin/perl install-module.pl DateTime::TimeZone
    /usr/bin/perl install-module.pl Template
    /usr/bin/perl install-module.pl Email::Send
    /usr/bin/perl install-module.pl Email::MIME
    /usr/bin/perl install-module.pl Email::MIME::Encodings
    /usr/bin/perl install-module.pl Email::MIME::Modifier
    /usr/bin/perl install-module.pl URI

To attempt an automatic install of every required and optional module with one command, do:

  /usr/bin/perl install-module.pl --all

6. Execute bugzilla install-module.pl

As suggested by the output of the checksetup.pl, you can execute the install-module.pl to install all bugzilla required and optional perl modules.
# /usr/bin/perl install-module.pl --all
Please review the output of the above install-module.pl to make sure everything got install properly. There is a possibility that some of the modules failed to install (may be because some required OS packages were missing).
Execute the checksetup.pl to verify whether all the modules got installed properly.
Following is the output of 2nd run of the checksetup.pl:
# ./checksetup.pl --check-modules
COMMANDS TO INSTALL OPTIONAL MODULES:

             GD: /usr/bin/perl install-module.pl GD
          Chart: /usr/bin/perl install-module.pl Chart::Base
    Template-GD: /usr/bin/perl install-module.pl Template::Plugin::GD::Image
     GDTextUtil: /usr/bin/perl install-module.pl GD::Text
        GDGraph: /usr/bin/perl install-module.pl GD::Graph
       XML-Twig: /usr/bin/perl install-module.pl XML::Twig
     PerlMagick: /usr/bin/perl install-module.pl Image::Magick
      SOAP-Lite: /usr/bin/perl install-module.pl SOAP::Lite
       mod_perl: /usr/bin/perl install-module.pl mod_perl2

YOU MUST RUN ONE OF THE FOLLOWING COMMANDS (depending on which database
you use):

PostgreSQL: /usr/bin/perl install-module.pl DBD::Pg
     MySQL: /usr/bin/perl install-module.pl DBD::mysql
    Oracle: /usr/bin/perl install-module.pl DBD::Oracle

7. Install missing Perl Modules

As we see from the above checksetup.pl output, some of the optional modules and required module installed was not completed when we ran the install-module.pl.
So, we have to install the missing modules manually one-by-one to figure out the issues and fix it one-by-one.
Refer to the “Troubleshooting Section” at the end for list of all the issues that I faced while installing the perl modules required for bugzilla (along with the solution on how to fix those issues).

8. Final checksetup.pl –check-modules verification

Execute checksetup.pl –check-modules again as shown below as final verification to make sure all the modules got installed successfully.
# ./checksetup.pl --check-modules
* This is Bugzilla 3.4.6 on perl 5.8.8
* Running on Linux 2.6.18-164.el5PAE #1 SMP Thu Sep 3 04:10:44 EDT 2009

Checking perl modules...
Checking for              CGI.pm (v3.21)   ok: found v3.49
Checking for          Digest-SHA (any)     ok: found v5.48
Checking for            TimeDate (v2.21)   ok: found v2.24
Checking for            DateTime (v0.28)   ok: found v0.55
Checking for   DateTime-TimeZone (v0.71)   ok: found v1.17
Checking for                 DBI (v1.41)   ok: found v1.52
Checking for    Template-Toolkit (v2.22)   ok: found v2.22
Checking for          Email-Send (v2.00)   ok: found v2.198
Checking for          Email-MIME (v1.861)  ok: found v1.903
Checking for Email-MIME-Encodings (v1.313)  ok: found v1.313
Checking for Email-MIME-Modifier (v1.442)  ok: found v1.903
Checking for                 URI (any)     ok: found v1.54 

Checking available perl DBD modules...
Checking for              DBD-Pg (v1.45)    not found
Checking for           DBD-mysql (v4.00)   ok: found v4.013
Checking for          DBD-Oracle (v1.19)    not found 

The following Perl modules are optional:
Checking for                  GD (v1.20)   ok: found v2.44
Checking for               Chart (v1.0)    ok: found v2.4.1
Checking for         Template-GD (any)     ok: found v1.56
Checking for          GDTextUtil (any)     ok: found v0.86
Checking for             GDGraph (any)     ok: found v1.44
Checking for            XML-Twig (any)     ok: found v3.34
Checking for          MIME-tools (v5.406)  ok: found v5.427
Checking for         libwww-perl (any)     ok: found v5.834
Checking for         PatchReader (v0.9.4)  ok: found v0.9.5
Checking for          PerlMagick (any)     ok: found v6.2.8
Checking for           perl-ldap (any)     ok: found v0.4001
Checking for         Authen-SASL (any)     ok: found v2.1401
Checking for          RadiusPerl (any)     ok: found v0.17
Checking for           SOAP-Lite (v0.710.06) ok: found v0.711
Checking for         HTML-Parser (v3.40)   ok: found v3.65
Checking for       HTML-Scrubber (any)     ok: found v0.08
Checking for Email-MIME-Attachment-Stripper (any)     ok: found v1.316
Checking for         Email-Reply (any)     ok: found v1.202
Checking for         TheSchwartz (any)     ok: found v1.10
Checking for      Daemon-Generic (any)     ok: found v0.61
Checking for            mod_perl (v1.999022) ok: found v2.000004

9. Create localconfig file using checksetup.pl

Execute checksetup.pl without any argument, which will create a localconfig file in the current directory. The localconfig file contains the key configuration parameters used by the bugzilla (for example, mysql db username and password).
# ./checksetup.pl
Reading ./localconfig...

This version of Bugzilla contains some variables that you may want to
change and adapt to your local settings. Please edit the file
./localconfig and rerun checksetup.pl.

The following variables are new to ./localconfig since you last ran
checksetup.pl:  create_htaccess, webservergroup, db_driver, db_host, db_name, db_user, db_pass, db_port, db_sock, db_check, index_html, cvsbin, interdiffbin, diffpath, site_wide_secret

10. Modify the localconfig file.

The only thing you need to modify the localconfig file is MySQL database db password by changing the $db_pass variable as shown below.
# vi ./localconfig
$db_pass = 'Bugs4All';

11. Modify /etc/my.cnf to increase bugzilla attachment size

Set the max_allowed_packet to 4M in the /etc/my.cnf to increase bugzilla attachment size.
# cat /etc/my.cnf
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
user=mysql
# Default to using old password format for compatibility with mysql 3.x
# clients (those using the mysqlclient10 compatibility package).
old_passwords=1

# Disabling symbolic-links is recommended to prevent assorted security risks;
# to do so, uncomment this line:
# symbolic-links=0

# Allow packets up to 4MB
max_allowed_packet=4M

[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
Restart the mysqld after this change.
# service mysqld restart

12. Create bugs mysql user

Add bugzilla user (bugs) to the mysql database as shown below.
# mysql -u root -p

mysql> GRANT SELECT, INSERT,
UPDATE, DELETE, INDEX, ALTER, CREATE, LOCK TABLES,
CREATE TEMPORARY TABLES, DROP, REFERENCES ON bugs.*
TO bugs@localhost IDENTIFIED BY 'Bugs4All';

mysql> FLUSH PRIVILEGES;

13. Create the bugzilla database

Execute the checksetup.pl (without any arguments) again to create the mysql bugzilla database. Since the localconfig file already exist, the second time when you execute the checksetup.pl, it will create the mysql database based on the information from localconfig file.
# ./checksetup.pl 

Creating database bugs...

Building Schema object from database...
Adding new table bz_schema ...
Initializing the new Schema storage...
Adding new table attach_data ...
Adding new table attachments ...
Adding new table bug_group_map ...
Adding new table bug_see_also ...
Adding new table bug_severity ...
Adding new table bug_status ...

Inserting values into the 'priority' table:
Inserting values into the 'bug_status' table:
Inserting values into the 'rep_platform' table:

Creating ./data directory...
Creating ./data/attachments directory...
Creating ./data/duplicates directory...

Adding foreign key: attachments.bug_id -> bugs.bug_id...
Adding foreign key: attachments.submitter_id -> profiles.userid...
Adding foreign key: bug_group_map.bug_id -> bugs.bug_id...

14. Create bugzilla administrator account.

At the end of the ./checksetup.pl execution, it will detect that you don’t have an adminsitrator account and request you to enter administration login information as shown below.
Looks like we don't have an administrator set up yet. Either this is
your first time using Bugzilla, or your administrator's privileges
might have accidentally been deleted.

Enter the e-mail address of the administrator: ramesh@thegeekstuff.com
Enter the real name of the administrator: Ramesh Natarajan
Enter a password for the administrator account: NotRealPwd
Please retype the password to verify: welcome
ramesh@thegeekstuff.com is now set up as an administrator.
Creating default classification 'Unclassified'...
Creating initial dummy product 'TestProduct'...

Now that you have installed Bugzilla, you should visit the 'Parameters'
page (linked in the footer of the Administrator account) to ensure it
is set up as you wish - this includes setting the 'urlbase' option to
the correct URL.

15. Configure apache for mod_perl

Rename the bugzilla directory. (i.e remove the version number in it)
# cd /var/www/html

# mv bugzilla-3.4.6/ bugzilla
Add the following two lines to httpd.conf
# tail -2 /etc/httpd/conf/httpd.conf
PerlSwitches -I/var/www/html/bugzilla -I/var/www/html/bugzilla/lib -w -T
PerlConfigRequire /var/www/html/bugzilla/mod_perl.pl
Verify the Group in httpd.conf matches the webservergroup in localconfig
# cd /var/www/html/bugzilla/

# grep webservergroup localconfig
$webservergroup = 'apache';

# grep Group /etc/httpd/conf/httpd.conf
Group apache

16. Final checksetup.pl execution

Execute the checksetup.pl again.
# ./checksetup.pl
Reading ./localconfig...

Removing existing compiled templates...
Precompiling templates...done.
Fixing file permissions...

Now that you have installed Bugzilla, you should visit the 'Parameters'
page (linked in the footer of the Administrator account) to ensure it
is set up as you wish - this includes setting the 'urlbase' option to
the correct URL.

17. Login to bugzilla and complete one time setup.

Start the apache, go to http://{your-ip-address}/bugzilla and login using the administrator account you created above.
From the bugzilla UI, at the footer -> Administration -> Parameters -> ‘Required Settings’ section -> Fill-out following information:
maintainer: ramesh@thegeekstuff.com
urlbase: http://{your-ip-address}/
Note: Depending on your setup, go to -> User Authentication -> and you might want to change requiredlogin and emailregexp parameter.

Troubleshooting Bugzilla Install Issues

Issue1: DBD::mysql module failed

The DBD:mysql perl module failed with the “mysql.h: No such file or directory” error message as shown below.
# /usr/bin/perl install-module.pl DBD::mysql

dbdimp.h:22:49: error: mysql.h: No such file or directory
dbdimp.h:23:45: error: mysqld_error.h: No such file or directory
dbdimp.h:25:49: error: errmsg.h: No such file or directory
In file included from dbdimp.c:20:
dbdimp.h:144: error: expected specifier-qualifier-list before âMYSQLâ
dbdimp.h:236: error: expected specifier-qualifier-list before âMYSQL_RESâ

Solution1: install mysql-devel

Error message “mysql.h: No such file or directory” is because mysql-devel package was missing as shown below.
# rpm -qa | grep -i mysql
MySQL-python-1.2.1-1
mysql-5.0.77-4.el5_4.2
mysql-connector-odbc-3.51.26r1127-1.el5
mysql-server-5.0.77-4.el5_4.2
libdbi-dbd-mysql-0.8.1a-1.2.2
perl-DBD-MySQL-3.0007-2.el5
Install the mysql-devel package as shown below.
# yum install mysql-devel

# rpm -qa | grep -i "mysql-devel"
mysql-devel-5.0.77-4.el5_4.2
DBD::mysql installation will go through without any issues now.
# /usr/bin/perl install-module.pl DBD::mysql

Issue2: GD failed with missing gdlib-config / libgd

Installing GD module failed with the following error message.
# /usr/bin/perl install-module.pl GD

**UNRECOVERABLE ERROR**
Could not find gdlib-config in the search path. Please install libgd 2.0.28 or higher.
If you want to try to compile anyway, please rerun this script with the option --ignore_missing_gd.
Running make test
  Make had some problems, maybe interrupted? Won't test
Running make install
  Make had some problems, maybe interrupted? Won't install

Solution2: Install gd-devel package

Install libgd (i.e gd-devel package) as shown below to fix the GD module issue.
# yum install gd-devel

# rpm -qa | grep gd
gd-2.0.33-9.4.el5_4.2
gd-devel-2.0.33-9.4.el5_4.2
GD got installed without any issues after insingalling gd-devel package.
# /usr/bin/perl install-module.pl GD

Issue3: Twig Failed with expat.h error

Twig module failed to install with the error message “expat.h: No such file or directory” as shown below.
# /usr/bin/perl install-module.pl XML::Twig

Expat.xs:12:19: error: expat.h: No such file or directory
Expat.xs:60: error: expected specifier-qualifier-list before XML_Parser

Solution3: Install expat and expat-devel for Twig

Install expat and expat-devel package as shown below.
# yum install expat

# yum install expat-devel
Now install Twig without any issues.
# /usr/bin/perl install-module.pl XML::Twig

Issue4: Image::Magick failed to install

Image::Magick installation failed with “magick/MagickCore.h: No such file or directory” error message as shown below.
# /usr/bin/perl install-module.pl Image::Magick

Note (probably harmless): No library found for -lMagickCore
Magick.xs:64:31: error: magick/MagickCore.h: No such file or directory
Magick.xs:171: error: expected specifier-qualifier-list before ‘MagickRealType’
Magick.xs:192: error: expected specifier-qualifier-list before ‘ImageInfo’
Magick.xs:214: error: ‘MagickNoiseOptions’ undeclared here (not in a function)
Magick.xs:214: warning: missing initializer

Solution4: Image::Magick failed to install

Make sure following ImageMagic related packages are present.
# rpm -qa | grep -i Image
ImageMagick-6.2.8.0-4.el5_1.1
ImageMagick-c++-devel-6.2.8.0-4.el5_1.1
ImageMagick-devel-6.2.8.0-4.el5_1.1
ImageMagick-c++-6.2.8.0-4.el5_1.1
ImageMagick-perl-6.2.8.0-4.el5_1.1
In my case, ImageMagic-devel was missing. So, installed it as shown below. After that, Image::Magick perl module got installed successfully.
# yum install ImageMagick-devel

# /usr/bin/perl install-module.pl Image::Magick

Issue5: SOAP::Lite failed to install

SOAP::Lite module failed to install with “Cannot locate version.pm in @INC” message as shown below.
#/usr/bin/perl install-module.pl SOAP::Lite

Failed test 'use SOAP::Lite;' at t/SOAP/Data.t line 5.
Tried to use 'SOAP::Lite'.
Error:  Can't locate version.pm in @INC

Solution5: Install version.pm required for SOAP::Lite

Installed version.pm as shown below. After this, SOAP::Lite got installed without any issue.
# perl -MCPAN -e 'install version' 

# /usr/bin/perl install-module.pl SOAP::Lite

Issue6 (and Solution6): mod_perl was missing

Don’t install mod_perl using /usr/bin/perl install-module.pl mod_perl2 . Insetad, use yum to install mod_perl as shown below.
# yum install mod_perl

Issue7: Apache start failed

Starting apache failed with “Cannot locate Template/Config.pm in @INC” error message.
# service httpd restart
Stopping httpd:                                            [  OK  ]

Starting httpd: Syntax error on line 994 of /etc/httpd/conf/httpd.conf:
Can't locate Template/Config.pm in @INC

Solution7: Install Template-Tool Kit as shown below

Install Template-Tool kit to fix the above apache error message
# cpan
cpan> i /Template-Toolkit/
Distribution    A/AB/ABEL/Eidolon-Driver-Template-Toolkit-0.01.tar.gz
Distribution    A/AB/ABW/Template-Toolkit-1.07.tar.gz
Distribution    A/AB/ABW/Template-Toolkit-2.22.tar.gz
Distribution    I/IN/INGY/Template-Toolkit-Simple-0.03.tar.gz
4 items found

cpan> install A/AB/ABW/Template-Toolkit-2.22.tar.gz

Issue8: Apache start failed again

Starting apache failed with “Cannot locate DateTime/Locale.pm in @INC” error message.
# service httpd restart
Stopping httpd:                                            [  OK  ]

Starting httpd: Syntax error on line 994 of /etc/httpd/conf/httpd.conf:
Can't locate DateTime/Locale.pm in @INC

Solution8: Install DateTime/Locale.pm as shown below

Install DateTime/Locale.pm to fix the above apache error message
# cpan

cpan> install DateTime:Locale
Also, in your apache error_log if you see Digest/SHA.pm issue, you should install it as shown below.
# tail -f /etc/httpd/logs/error_log
Can't locate Digest/SHA.pm in @INC (@INC contains: 

# cpan
cpan> install Digest::SHA
 
 
 
------------------------------------------
 

If you enjoyed this article, you might also like..

  1. 50 Linux Sysadmin Tutorials
  2. 50 Most Frequently Used Linux Commands (With Examples)
  3. Top 25 Best Linux Performance Monitoring and Debugging Tools
  4. Mommy, I found it! – 15 Practical Linux Find Command Examples
  5. Linux 101 Hacks 2nd Edition eBook Linux 101 Hacks Book
 
 
 
 

Thursday, October 17, 2013

Automating a JBoss Deployment with ANT

http://www.brainhemorage.com/?p=149

One area where virtually all organizations seem to differ is in change management–and specifically in the deployment strategy.  Recently, a client requested I put together deployment instructions, specifying a preference that I include a deployment script that could remotely deploy the application.  To meet this requirement, I decided to incorporate this new functionality into my existing ANT build script.
In my case, the application is a JBoss Web Service deployment.  The following steps are necessary to deploy the application:
  1. Stop the JBoss Server.
  2. Removed the previous EAR deployment.
  3. Copy the new EAR file up to the server.
  4. Restart the JBoss Server.
Note that restarting the JBoss server is not entirely necessary for many applications.  The JBoss hot deploy feature will gladly pick up the changes. However, we’ve had a few issues with hot deployments in other applications and over time have found it best to restart the application server for each deployment. Your mileage may vary.
Also, because we will eventually deploy to multiple environments, I want to be able to store configuration parameters for each target environment in its own .properties file.
The expected command line to run this deployment will be:
ant -Denv=targetEnvironment jboss-deploy
Where targetEnvironment specifies the target deployment environment.
To get started, I added an ANT target to verify two things:
  • That an environment has been specified
  • The corresponding env.properties file exists.
If these conditions are met, it reads in the target environment’s configution properties.  Here is the code:
<target name="-jboss-deploy-prep">
  <fail
     message="Target environment must be specified in ant command line as -Denv=target"
     unless="env" />

  <fail message="conf/${env}.properties file does not exist">
    <condition>
      <not>
        <available file="conf/${env}.properties" />
      </not>
    </condition>
  </fail>

  <!-- Get properties for target environment -->
  <property file="conf/${env}.properties" />
</target>
The contents of the properties file goes something like this:
jboss.remote.instance.home=/home/jboss/jboss-5.1.0.GA/server/testapp
jboss.remote.host=myhostname
jboss.remote.username=jboss
#jboss.remote.password=secret
jboss.remote.stop=/home/jboss/jboss-5.1.0.GA/server/testapp/bin/testapp stop
jboss.remote.start=/home/jboss/jboss-5.1.0.GA/server/testapp/bin/testapp start
Note that the password is commented out. For security reasons, we may not want to keep the password in the properties file. If the password is not specified, we want to prompt for it. So I added this target:
<!-- get password if not specified -->
<target name="-jboss-prompt-password" unless="jboss.remote.password">
  <input message="Jboss Remote Password:" addproperty="jboss.remote.password">
    <handler classname="org.apache.tools.ant.input.SecureInputHandler" />
  </input>
</target>
Now that I have all the right pieces in place, time to add the jboss-deploy target:
<target
   name="jboss-deploy"
   depends="clean, dist-app, -jboss-deploy-prep, -jboss-prompt-password" >

  <echo message="JBoss Deployment" />
  <echo message="--------------------------------------------------------" />

  <echo message="Stopping the JBoss server running in ${env}" />
  <sshexec
     trust="true"
     host="${jboss.remote.host}"
     username="${jboss.remote.username}" password="${jboss.remote.password}"
     command="${jboss.remote.stop}" />

  <echo message="Removing existing deployment ..." />
  <sshexec
     trust="true"
     host="${jboss.remote.host}"
     username="${jboss.remote.username}" password="${jboss.remote.password}"
     command="rm -f ${jboss.remote.instance.home}/deploy/rce-j2ee-*.ear" />

  <echo message="Transferring new deployment ..." />
  <scp
    sftp="true" trust="true"
    file="${file.appdist}"
    todir="${jboss.remote.username}@${jboss.remote.host}:${jboss.remote.instance.home}/deploy"
    password="${jboss.remote.password}" />

  <echo message="Starting JBoss server running in ${env}" />
  <sshexec
    trust="true"
    host="${jboss.remote.host}"
    username="${jboss.remote.username}" password="${jboss.remote.password}"
    command="${jboss.remote.start}" />

</target>
That’s it. Now executing the jboss-deploy target does the following:
  • Compiles my application
  • Creates a versioned EAR file
  • Stops the JBoss Server
  • Removes the previous deployment
  • Securely copies the new EAR to the server
  • Starts the JBoss server
One other note: if your start/stop scripts make use of the sudo command (as mine does), you may need to modify your sudoers configuration to skipping checking for TTY. The sshexec task does not allocate a TTY terminal, so many Linux distros will not allow the sudo.

autostart JBOSS when power up

http://www.blog.rogersit.net/simple-jboss-7-server-startup-script-linux/2012/01/19/

#!/bin/sh
#chkconfig: 345 99 10
#description: JBoss auto start-stop script.
# Source function library.
. /etc/rc.d/init.d/functions
# Get config.
. /etc/sysconfig/network
# Check that networking is up.
[ "${NETWORKING}" = "no" ] && exit 0
### CHANGE THE STARTUP PATH TO YOUR START SCRIPT ###
startup='/home/jboss/jboss-as-web-7.0.2.Final/bin/standalone.sh > /dev/null 2> /dev/null &'
shutdown='killall java'
start(){
echo -n $"Starting JBoss service: "
$startup
RETVAL=$?
echo
}
stop(){
action $"Stopping JBoss service: " $shutdown
RETVAL=$?
echo
}
restart(){
stop
sleep 10
start
}
# See how we were called.
case "$1" in
start)
start
;;
stop)
stop
;;
restart)
restart
;;
*)
echo $"Usage: $0 {start|stop|restart}"
exit 1
esac
1: Name and add it to /etc/init.d
2: chmod 755 jboss
3: chkconfig --add jboss && chkconfig jboss on

Command Usage
service jboss start
service jboss stop
servoce jboss restart

Monday, August 26, 2013

SVN svnsync recreate

If you have some specific problem with a known good svn mirror, i.e. you can no longer checkout files from this mirror due to errors like url was changed, uuid not matched, etc. run svnsync switch:

svn propset --revprop -r 0 svn:sync-from-url http://my.new/url url://to.my.repo
for example, provided the mirror is local:
svn propset --revprop --username  myname -r 0 svn:sync-from-url svn://xx.xx.xxx.xx:9999  file:///var/www/svn/svn_mirror

Now you can run svnsync to sync from the new URL
svnsync –non-interactive sync --username=<name> url://to.my.repo
for example:
/usr/bin/svnsync sync --non-interactive --username myname  file:///var/www/svn/svn_mirror


And for whatever reason, worst case scenario is if you have to recreate the mirror:
1. move current mirror
    mv  /var/www/svn/<repo1>   /var/www/svn/<repo1>.OLD
2. create new repo with same name
    svnadmin create /var/www/svn/<repo1>
    chown -R apache  /var/www/svn/<repo1>
    chgrp  -R labtools  /var/www/svn/<repo1>
3. copy pre-revprop-change to preserve the auth.
    cp /var/www/svn/<repo1>.OLD/hooks/pre-revprop-change  /var/www/svn/<repo1>/hooks/
4. init the mirror
    svnsync init --username <user>   file:///var/www/svn/<repo1>  svn://<svn repository>
5. sync the mirror
    svnsync sync --username <user>   file:///var/www/svn/<repo1>
6. delete old mirror
    rm  -rf  /var/www/svn/<repo1>.OLD



--------------------------------
FYI: subversion server setup RHEL and centOS:

install apache server and php:
yum --enablerepo=remi install httpd php php-common

install dependencies:
yum install php-pecl-apc php-cli php-pear php-pdo php-mysql php-pgsql php-pecl-mongo php-sqlite php-pecl-memcache php-pecl-memcached php-gd php-mbstring php-mcrypt php-xml

start apache:
/etc/init.d/httpd start ## use restart after update
## OR ##
service httpd start ## use restart after update

chkconfig --levels 235 httpd on

add this /var/www/html/test.php file
<?php
    phpinfo();
?>

install subversion:
yum install mod_dav_svn subversion

create svn directory:
mkdir /var/www/svn
cd /var/www/svn

svnadmin create repos1
chown -R apache.apache repo1

chcon -R -t httpd_sys_content_t /var/www/svn/repo1

## Following enables commits over http ##
chcon -R -t httpd_sys_rw_content_t /var/www/svn/repo1


modify Subversion config file /etc/httpd/conf.d/subversion.conf

LoadModule dav_svn_module     modules/mod_dav_svn.so
LoadModule authz_svn_module   modules/mod_authz_svn.so
<Location /svn>
   DAV svn
   SVNParentPath /var/www/svn
   AuthType Basic
   AuthName "Subversion repositories"
   AuthUserFile /etc/svn-auth-users
   Require valid-user
</Location>

## Create testuser ##
htpasswd -cm /etc/svn-auth-users testuser
New password:
Re-type new password:
Adding password for user testuser

## Create testuser2 ##
htpasswd -m /etc/svn-auth-users testuser2
New password:
Re-type new password:
Adding password for user testuser2

always restart httpd when made change to config file
/etc/init.d/httpd restart

Tuesday, July 23, 2013

svn some commandlines switches


1. to preserve check-in date when checkout:

svn checkout --config-option config:miscellany:use-commit-times=yes

2. in jenkins, to check out several directories in different places:


Assuming you're using Subversion to checkout your projects, you need to specify the "Local module directory" to something other than the default for each path you are checking out.
For example;
If you have svn://myrepo/myproject/plugins and svn://myrepo/myproject/tests, the configuration would be along the lines of;
Modules:
Repository URL : svn://myrepo/myproject/plugins
Local module directory (optional) : plugins

Repository URL : svn://myrepo/myproject/tests
Local module directory (optional) : tests
This would then inform Jenkins that it has two paths to checkout and into separate locations.
If you are trying to checkout a project into the subfolder of another working copy, you may have to use svn:externals on the parent directory.
3. for svn error "svnsync failed to get lock on destination repos currently held by user ..." it had to delete the lock in order to make it work again "svn propdel --revprop -r0 svn:sync-lock file:///var/www/svn/repository-1 "

Tuesday, July 16, 2013

MySQL Commands

http://www.pantz.org/software/mysql/mysqlcommands.html

Below when you see # it means from the unix shell. When you see mysql> it means from a MySQL prompt after logging into MySQL.

To login (from unix shell) use -h only if needed.

# [mysql dir]/bin/mysql -h hostname -u root -p

Create a database on the sql server.

mysql> create database [databasename];

List all databases on the sql server.

mysql> show databases;

Switch to a database.

mysql> use [db name];

To see all the tables in the db.

mysql> show tables;

To see database's field formats.

mysql> describe [table name];

To delete a db.

mysql> drop database [database name];

To delete a table.

mysql> drop table [table name];

Show all data in a table.

mysql> SELECT * FROM [table name];

Returns the columns and column information pertaining to the designated table.

mysql> show columns from [table name];

Show certain selected rows with the value "whatever".

mysql> SELECT * FROM [table name] WHERE [field name] = "whatever";

Show all records containing the name "Bob" AND the phone number '3444444'.

mysql> SELECT * FROM [table name] WHERE name = "Bob" AND phone_number = '3444444';

Show all records not containing the name "Bob" AND the phone number '3444444' order by the phone_number field.

mysql> SELECT * FROM [table name] WHERE name != "Bob" AND phone_number = '3444444' order by phone_number;

Show all records starting with the letters 'bob' AND the phone number '3444444'.

mysql> SELECT * FROM [table name] WHERE name like "Bob%" AND phone_number = '3444444';

Show all records starting with the letters 'bob' AND the phone number '3444444' limit to records 1 through 5.

mysql> SELECT * FROM [table name] WHERE name like "Bob%" AND phone_number = '3444444' limit 1,5;

Use a regular expression to find records. Use "REGEXP BINARY" to force case-sensitivity. This finds any record beginning with a.

mysql> SELECT * FROM [table name] WHERE rec RLIKE "^a";

Show unique records.

mysql> SELECT DISTINCT [column name] FROM [table name];

Show selected records sorted in an ascending (asc) or descending (desc).

mysql> SELECT [col1],[col2] FROM [table name] ORDER BY [col2] DESC;

Return number of rows.

mysql> SELECT COUNT(*) FROM [table name];

Sum column.

mysql> SELECT SUM(*) FROM [table name];

Join tables on common columns.

mysql> select lookup.illustrationid, lookup.personid,person.birthday from lookup left join person on lookup.personid=person.personid=statement to join birthday in person table with primary illustration id;

Creating a new user. Login as root. Switch to the MySQL db. Make the user. Update privs.

# mysql -u root -p
mysql> use mysql;
mysql> INSERT INTO user (Host,User,Password) VALUES('%','username',PASSWORD('password'));
mysql> flush privileges;

Change a users password from unix shell.

# [mysql dir]/bin/mysqladmin -u username -h hostname.blah.org -p password 'new-password'

Change a users password from MySQL prompt. Login as root. Set the password. Update privs.

# mysql -u root -p
mysql> SET PASSWORD FOR 'user'@'hostname' = PASSWORD('passwordhere');
mysql> flush privileges;

Recover a MySQL root password. Stop the MySQL server process. Start again with no grant tables. Login to MySQL as root. Set new password. Exit MySQL and restart MySQL server.

# /etc/init.d/mysql stop
# mysqld_safe --skip-grant-tables &
# mysql -u root
mysql> use mysql;
mysql> update user set password=PASSWORD("newrootpassword") where User='root';
mysql> flush privileges;
mysql> quit
# /etc/init.d/mysql stop
# /etc/init.d/mysql start

Set a root password if there is on root password.

# mysqladmin -u root password newpassword

Update a root password.

# mysqladmin -u root -p oldpassword newpassword

Allow the user "bob" to connect to the server from localhost using the password "passwd". Login as root. Switch to the MySQL db. Give privs. Update privs.

# mysql -u root -p
mysql> use mysql;
mysql> grant usage on *.* to bob@localhost identified by 'passwd';
mysql> flush privileges;

Give user privilages for a db. Login as root. Switch to the MySQL db. Grant privs. Update privs.

# mysql -u root -p
mysql> use mysql;
mysql> INSERT INTO db (Host,Db,User,Select_priv,Insert_priv,Update_priv,Delete_priv,Create_priv,Drop_priv) VALUES ('%','databasename','username','Y','Y','Y','Y','Y','N');
mysql> flush privileges;

or

mysql> grant all privileges on databasename.* to username@localhost;
mysql> flush privileges;

To update info already in a table.

mysql> UPDATE [table name] SET Select_priv = 'Y',Insert_priv = 'Y',Update_priv = 'Y' where [field name] = 'user';

Delete a row(s) from a table.

mysql> DELETE from [table name] where [field name] = 'whatever';

Update database permissions/privilages.

mysql> flush privileges;

Delete a column.

mysql> alter table [table name] drop column [column name];

Add a new column to db.

mysql> alter table [table name] add column [new column name] varchar (20);

Change column name.

mysql> alter table [table name] change [old column name] [new column name] varchar (50);

Make a unique column so you get no dupes.

mysql> alter table [table name] add unique ([column name]);

Make a column bigger.

mysql> alter table [table name] modify [column name] VARCHAR(3);

Delete unique from table.

mysql> alter table [table name] drop index [colmn name];

Load a CSV file into a table.

mysql> LOAD DATA INFILE '/tmp/filename.csv' replace INTO TABLE [table name] FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' (field1,field2,field3);

Dump all databases for backup. Backup file is sql commands to recreate all db's.

# [mysql dir]/bin/mysqldump -u root -ppassword --opt >/tmp/alldatabases.sql

Dump one database for backup.

# [mysql dir]/bin/mysqldump -u username -ppassword --databases databasename >/tmp/databasename.sql

Dump a table from a database.

# [mysql dir]/bin/mysqldump -c -u username -ppassword databasename tablename > /tmp/databasename.tablename.sql

Restore database (or database table) from backup.

# [mysql dir]/bin/mysql -u username -ppassword databasename < /tmp/databasename.sql

Create Table Example 1.

mysql> CREATE TABLE [table name] (firstname VARCHAR(20), middleinitial VARCHAR(3), lastname VARCHAR(35),suffix VARCHAR(3),officeid VARCHAR(10),userid VARCHAR(15),username VARCHAR(8),email VARCHAR(35),phone VARCHAR(25), groups VARCHAR(15),datestamp DATE,timestamp time,pgpemail VARCHAR(255));

Create Table Example 2.

mysql> create table [table name] (personid int(50) not null auto_increment primary key,firstname varchar(35),middlename varchar(50),lastnamevarchar(50) default 'bato');
MYSQL Statements and clauses
ALTER DATABASE

ALTER TABLE

ALTER VIEW

ANALYZE TABLE

BACKUP TABLE

CACHE INDEX

CHANGE MASTER TO

CHECK TABLE

CHECKSUM TABLE

COMMIT

CREATE DATABASE

CREATE INDEX

CREATE TABLE

CREATE VIEW

DELETE

DESCRIBE

DO

DROP DATABASE

DROP INDEX

DROP TABLE

DROP USER

DROP VIEW

EXPLAIN

FLUSH

GRANT

HANDLER

INSERT

JOIN

KILL

LOAD DATA FROM MASTER

LOAD DATA INFILE

LOAD INDEX INTO CACHE

LOAD TABLE...FROM MASTER

LOCK TABLES

OPTIMIZE TABLE

PURGE MASTER LOGS

RENAME TABLE

REPAIR TABLE

REPLACE

RESET

RESET MASTER

RESET SLAVE

RESTORE TABLE

REVOKE

ROLLBACK

ROLLBACK TO SAVEPOINT

SAVEPOINT

SELECT

SET

SET PASSWORD

SET SQL_LOG_BIN

SET TRANSACTION

SHOW BINLOG EVENTS

SHOW CHARACTER SET

SHOW COLLATION

SHOW COLUMNS

SHOW CREATE DATABASE

SHOW CREATE TABLE

SHOW CREATE VIEW

SHOW DATABASES

SHOW ENGINES

SHOW ERRORS

SHOW GRANTS

SHOW INDEX

SHOW INNODB STATUS

SHOW LOGS

SHOW MASTER LOGS

SHOW MASTER STATUS

SHOW PRIVILEGES

SHOW PROCESSLIST

SHOW SLAVE HOSTS

SHOW SLAVE STATUS

SHOW STATUS

SHOW TABLE STATUS

SHOW TABLES

SHOW VARIABLES

SHOW WARNINGS

START SLAVE

START TRANSACTION

STOP SLAVE

TRUNCATE TABLE

UNION

UNLOCK TABLES

USE

String Functions
AES_DECRYPT

AES_ENCRYPT

ASCII

BIN

BINARY

BIT_LENGTH

CHAR

CHAR_LENGTH

CHARACTER_LENGTH

COMPRESS

CONCAT

CONCAT_WS

CONV

DECODE

DES_DECRYPT

DES_ENCRYPT

ELT

ENCODE

ENCRYPT

EXPORT_SET

FIELD

FIND_IN_SET

HEX

INET_ATON

INET_NTOA

INSERT

INSTR

LCASE

LEFT

LENGTH

LOAD_FILE

LOCATE

LOWER

LPAD

LTRIM

MAKE_SET

MATCH    AGAINST

MD5

MID

OCT

OCTET_LENGTH

OLD_PASSWORD

ORD

PASSWORD

POSITION

QUOTE

REPEAT

REPLACE

REVERSE

RIGHT

RPAD

RTRIM

SHA

SHA1

SOUNDEX

SPACE

STRCMP

SUBSTRING

SUBSTRING_INDEX

TRIM

UCASE

UNCOMPRESS

UNCOMPRESSED_LENGTH

UNHEX

UPPER

Date and Time Functions
ADDDATE

ADDTIME

CONVERT_TZ

CURDATE

CURRENT_DATE

CURRENT_TIME

CURRENT_TIMESTAMP

CURTIME

DATE

DATE_ADD

DATE_FORMAT

DATE_SUB

DATEDIFF

DAY

DAYNAME

DAYOFMONTH

DAYOFWEEK

DAYOFYEAR

EXTRACT

FROM_DAYS

FROM_UNIXTIME

GET_FORMAT

HOUR

LAST_DAY

LOCALTIME

LOCALTIMESTAMP

MAKEDATE

MAKETIME

MICROSECOND

MINUTE

MONTH

MONTHNAME

NOW

PERIOD_ADD

PERIOD_DIFF

QUARTER

SEC_TO_TIME

SECOND

STR_TO_DATE

SUBDATE

SUBTIME

SYSDATE

TIME

TIMEDIFF

TIMESTAMP

TIMESTAMPDIFF

TIMESTAMPADD

TIME_FORMAT

TIME_TO_SEC

TO_DAYS

UNIX_TIMESTAMP

UTC_DATE

UTC_TIME

UTC_TIMESTAMP

WEEK

WEEKDAY

WEEKOFYEAR

YEAR

YEARWEEK

Mathematical and Aggregate Functions
ABS

ACOS

ASIN

ATAN

ATAN2

AVG

BIT_AND

BIT_OR

BIT_XOR

CEIL

CEILING

COS

COT

COUNT

CRC32

DEGREES

EXP

FLOOR

FORMAT

GREATEST

GROUP_CONCAT

LEAST

LN

LOG

LOG2

LOG10

MAX

MIN

MOD

PI

POW

POWER

RADIANS

RAND

ROUND

SIGN

SIN

SQRT

STD

STDDEV

SUM

TAN

TRUNCATE

VARIANCE

Flow Control Functions
CASE

IF

IFNULL

NULLIF

Command-Line Utilities
comp_err

isamchk

make_binary_distribution

msql2mysql

my_print_defaults

myisamchk

myisamlog

myisampack

mysqlaccess

mysqladmin

mysqlbinlog

mysqlbug

mysqlcheck

mysqldump

mysqldumpslow

mysqlhotcopy

mysqlimport

mysqlshow

perror

Perl API - using functions and methods built into the Perl DBI with MySQL
available_drivers

begin_work

bind_col

bind_columns

bind_param

bind_param_array

bind_param_inout

can

clone

column_info

commit

connect

connect_cached

data_sources

disconnect

do

dump_results

err

errstr

execute

execute_array

execute_for_fetch

fetch

fetchall_arrayref

fetchall_hashref

fetchrow_array

fetchrow_arrayref

fetchrow_hashref

finish

foreign_key_info

func

get_info

installed_versions


last_insert_id

looks_like_number

neat

neat_list

parse_dsn

parse_trace_flag

parse_trace_flags

ping

prepare

prepare_cached

primary_key

primary_key_info

quote

quote_identifier

rollback

rows

selectall_arrayref

selectall_hashref

selectcol_arrayref

selectrow_array

selectrow_arrayref

selectrow_hashref

set_err

state

table_info

table_info_all

tables

trace

trace_msg

type_info

type_info_all

Attributes for Handles

PHP API - using functions built into PHP with MySQL
mysql_affected_rows

mysql_change_user

mysql_client_encoding

mysql_close

mysql_connect

mysql_create_db

mysql_data_seek

mysql_db_name

mysql_db_query

mysql_drop_db

mysql_errno

mysql_error

mysql_escape_string

mysql_fetch_array

mysql_fetch_assoc

mysql_fetch_field

mysql_fetch_lengths

mysql_fetch_object

mysql_fetch_row

mysql_field_flags

mysql_field_len

mysql_field_name

mysql_field_seek

mysql_field_table

mysql_field_type

mysql_free_result

mysql_get_client_info

mysql_get_host_info

mysql_get_proto_info

mysql_get_server_info

mysql_info

mysql_insert_id

mysql_list_dbs

mysql_list_fields

mysql_list_processes

mysql_list_tables

mysql_num_fields

mysql_num_rows

mysql_pconnect

mysql_ping

mysql_query

mysql_real_escape_string

mysql_result

mysql_select_db

mysql_stat

mysql_tablename

mysql_thread_id

mysql_unbuffered_query

Monday, July 15, 2013

FAQs: svn/maven/ant/elcipse/


Subversion FAQ:
http://subversion.apache.org/faq.html

maven technical FAQ
http://maven.apache.org/general.html

maven Eclipse FAQ
http://maven.apache.org/maven-1.x/plugins/eclipse/faq.html

maven ant FAQ
http://ant.apache.org/faq.html

maven ANT tasks FAQ
http://maven.apache.org/ant-tasks/

Eclipse FAQ
http://wiki.eclipse.org/index.php/Eclipse_FAQs














Continuous Integration with Jenkins and Xcode

http://grayunicorn.com/?p=91

Last week (15 November 2012) I gave a short talk on setting up a continuous integration environment with Jenkins at Melbourne Cocoaheads. This is pretty much the content, converted into a blog post. Some said afterwards they would like more detail on various parts, so here is the whole thing with the shell script at the end. I hope you find it useful. I have to give a lot of thanks to all the other bloggers who wrote about their own experiences in doing this and to Stack Overflow.

MOTIVATION

First up, why would you do this? There are two big problems that this setup solves. First and most broadly speaking, continuous integration solves the problem of a broken build where nobody really knows how it got broken, which change broke it or how long it has been like that. If you let your project get into that state then it will be a miracle if you ever deliver anything, dealing with this issue is a huge part of success when working in a team. Because the build is performed every time anyone submits to the repository you will know right away that there is a problem and you can fix it right away. You can also apply some appropriate penalty to the person that did it – they can buy the team a coffee or something,pour encourager les autres1.
Secondly and more closely tied in with iOS development is that you can have your automated build sign and deploy a build so that anyone can point their iOS device at your distribution server with mobile Safari and have it install your build using Over The Air (OTA) provisioning. This is a feature of iOS available since release 4. You, the developer, will no longer have to deal with Important People tapping you on the shoulder asking for a build – you can direct them to your server. Remote testers can easily grab a build too.
There are some services that do this for you. Two I know of are Hockey and Testflight but this article deals with neither. They work well and provide a lot of extra features, particularly around bug and tester management, but it’s good to know what’s going on underneath. When you’ve done this setup you may choose to move on to one of those services for those extra features.

BUILD ON SUCCESS

Before getting started with this process, make sure you can do a successful Archive build in your regular work environment. If you can’t then it isn’t going to be any easier attempting to do it headless and automatically. That means all private keys, developer certificates, provisioning profiles and Xcode functioning together happily – sometimes this is not a trivial task. As a little refresher, here’s how all the developer pieces come together in code signing.
Screen Shot 2012 11 22 at 1 40 59 PM

VIRTUALLY RISK-FREE

Having configured Jenkins and OTA provisioning once or twice before I hesitated to install Jenkins and jump into command line changes on the machine I work on every day. Instead I installed a clean install of OS X 10.8 (Mountain Lion) using VMware Fusion, version 4.1.4 – couldn’t be easier. OS X came from the Mac App Store, the .dmg was converted to a CDR and VMware treated it as a boot CD. According to the Mac OS X 10.8.2 EULA, you are licensed to do this if you bought your copy from the Mac App Store.2
In addition to not potentially screwing up the machine on which you earn a living, virtualisation allows you to make a snapshot of your Virtual Machine (VM) – a saved game, if you like. Any time you know you have got something right you can pause the machine and take another snapshot. Any time things go badly wrong you can revert. The cost of this is nothing but disk space.
First up, the new VM needs Xcode installed. When done it is a good idea to license Xcode for all the users on the machine using the terminal:
sudo xcodebuild -license
The reason for this is that your build system will not be interactive, it can’t pop up the click-to-accept license dialog. About 7,109 presses of the space bar will be required before you can type “agree” to accept the license for all users on the system. I would never advocate using ‘G’ to make less skip straight to the end of the license agreement, you should read it all. (This is the first of many useful bits of information from an excellent session from WWDC 2012, hereafter simply called “404″3)
Next up, java. Theoretically java should be installed for you automatically when Jenkins tries to start. Once I installed Jenkins and it did not start, reason unknown. Now I type “java” at the command prompt and cause it to be installed before installing Jenkins. This will download and install java from the Mac App Store, nice and easy.
Before installing Jenkins I like to create the “jenkins” user. If you install Jenkins first the user is created but not as a regular, interactive user. Creating the user will make it easier to deal with some keychain operations later. You can create this user in the normal way with System Preferences. I made “jenkins” an administrator but it probably does not have to be.
Finally, time to install Jenkins itself. Because of my VM strategy I downloaded the Jenkins package on the host machine then copied it into the VM via a shared folder. If things go wrong this will save you from having to download it again. The Jenkins installer is fairly simple and nothing needs to be changed though you might want to look at the customizebutton to check that settings have not changed since I wrote this. Jenkins should be made to run as “jenkins”, the user you just created.
Following a successful Jenkins installation the Jenkins console should pop up in your browser – but this isn’t successful just yet. As of version 1.489, Jenkins includes a property list that must be edited first. At the console,
sudo launctl unload /Library/LoadDaemons/org.jenkins-ci.plist
then edit it to point to user “jenkins” actual home folder, (/Users/jenkins) and
sudo launctl load /Library/LoadDaemons/org.jenkins-ci.plist
Screen Shot 2012 11 22 at 1 48 44 PMJenkins is now configured to load when the system starts and should be running! If you launch a browser and enter http://127.0.0.1:8080 you should see the Jenkins console, ready to go to work.
In your clean system + Xcode you have git installed, but Jenkins can’t find it yet. You need to go to “Manage Jenkins” and specify the path. That path will be
/Applications/Xcode.app/Contents/Developer/usr/bin/git.
Jenkins supports SVN as installed, but I’m using git. For git users, you will want to install the Jenkins Git plugin. Github users will also want to install GitHub and GitHub API plugins. The way Jenkins uses git is to clone the repository you want to build before each build commences. I guess you could do this with a simple copy or ftp also, but I hope you are using some kind of revision control.
Before configuring the Jenkins build job, user “jenkins” needs a bit of setup. In your day-to-day environment, the one where you can successfully build an Archive of your project for distribution – remember that “Build on Success” section? – you have a keychain containing your developer credentials, and “jenkins” does not. You need to export them and then import them into a keychain for jenkins to use during the build. Your private key and your distribution certificate are required to make an OTA build, so export those and transfer them into the jenkins account using your public folder or similar.
A note about privacy – your private key is private, and important. Don’t leave it in your public folder. There is discussion on StackOverflow about how sharing a key like this is actually a bad idea, but I don’t know how to do it better.
Switch to the jenkins account and it’s time for some terminal magic:
security create-keychain -p jenkins JenkinsCI
security default-keychain -s JenkinsCI
security import -k JenkinsCI -P security import -k JenkinsCI
These commands create a new keychain named “JenkinsCI”, set it to be the default keychain, and then import the private key and distribution certificate you have transferred in to this account. The password “jenkins” is used throughout. During the build these assets will be used to sign the code with your developer credentials.
The provisioning profile is the last developer artefact needed. In your own account you will find it in ~/Library/MobileDevice/Provisioning Profiles, where Xcode puts it. You might choose to put that in the same place in the jenkins account, or you might choose to check it in with the code in the repository. Either way it must be available during the final stage of the build.
Now the configuration of the Jenkins job can begin. Each job in Jenkins is a series of steps that you define to produce some useful outcome. Here the outcome is a downloadable, installable iOS package on an accessible web server. Start by clicking “New Job” in the Jenkins console.

NEW JOB

This job is configured as a “Free-style software project”. A Multi-configuration project is shown in 4043 but this is only slightly different, a little simpler. There are six main phases to configure and once you have chosen a name for this job you are done with stage one. It remains to fetch code from the repository, build it, sign it and upload it. Finally you will set up when the job will execute.

FETCH

Assuming you are a git user, in the Source Code Management section of job configuration you can just select git and then enter the repo location, such ashttps://github.com/[GitHub account]/[Project Name].git/ – on every project in your GitHub repository you will find the clone path at the top of the repo page. If your repos are not public you will need to deal with creating an ssh key for your jenkins user in your GitHub account4 – or use HTTPS, or any method that works your your repo. At the start of every job execution Jenkins will clone the entire git repository and use it for the build.

BUILD

At this point it is time to be sure that jenkins has access to the keychain created for this build. In the Build section of job configuration you need to add a build step, “Execute shell”. You’ll get a window that accepts shell commands which will be run. The first command to enter is
security unlock-keychain -p jenkins JenkinsCI
As you might guess this unlocks the keychain, allowing the rest of the script access to developer artefacts as required.
In this script one very important variable set for you here is $WORKSPACE – it is set by Jenkins and is your pointer into the place where everything happens. It is used many times in the build commands.
Before building with Xcode in Jenkins, another important variable should be set. Thexcode-select command sets the default installation of Xcode that will be used. TheDEVELOPER_DIR variable can override xcode-select, so it is a good idea to be sure it is set to the right value before proceeding even though in this case there is only one Xcode installed. In future you might lift this script to run on some machine with more than one.
The build itself is deceptively simple:
xcodebuild \ 
    -project [Project Name].xcodeproj \ 
    -target [Target Name] \ 
    -configuration Release \ 
    -sdk iphoneos \ 
    CODE_SIGN_IDENTITY="${SIGNING_IDENTITY}"
The command xcodebuild is the main command used to execute builds from the command line with Xcode. My project is a project rather than a workspace, so I am specifying the .xcodeproj rather than the .workspace here. The target is the target name as you would see it in Xcode. The configuration is Release (but it could be Debug) and the sdk is iPhone rather than simulator.
Initially I had a command that looked a lot more like what is presented in 4043 but I got an error message that indicated the code signing identity was not set. Why this happened I do not know, but specifying the code signing identity, like
SIGNING_IDENTITY="iPhone Distribution: [Developer Name]" fixed it. Now Xcode is building your code.

SIGN

When the build has finished you have a .app file, which isn’t very useful. Because it is built for the iphoneos sdk it doesn’t run on the simulator and because it isn’t signed it doesn’t run on a device. Let’s sign it.
The other command used with Xcode on the command line is xcrun – it is a two-phase thing, finding and then executing commands included with the Xcode distribution. To sign,xcrun is using a perl script included with Xcode called PackageApplication.
xcrun -sdk iphoneos PackageApplication \
    -o "${WORKSPACE}/[Target Name].ipa" \
    -verbose "${WORKSPACE}/build/Release-iphoneos/[Target Name].app" \
    -sign "${SIGNING_IDENTITY}" \
    --embed "${PROVISIONING_PROFILE_DIR}/${PROVISIONING_PROFILE}"
It can take a lot of options. The ones I found necessary are the output, where you specify where the .ipa archive will be place. The input, which is the .app built in the xcodebuildstage. The signing identity, also the same as the xcodebuild stage. Finally the provisioning profile, which you may have copied into the jenkins account or you may have committed with your code.
At this stage you may get an error – I did. Some searching revealed the cause of the error is a misconfigured Xcode installation, a soft link that should be present is not. The message is “Object file format invalid or unsuitable” which may lead you to think that the failure has something to do with the object file format. Bad error message!
It is fixed with this variable set in the script:
export CODESIGN_ALLOCATE=/Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/codesign_allocate
Then – happiness. You should see “Finished: SUCCESS” in the Jenkins console output, and if you have been executing the job along with the description then the ball will be blue.

UPLOAD

After all that, OTA deployment is easy. There are three parts necessary before you can get a successful download to a device. The html, which must include a link with the itms-services:scheme, linking to the property list; the property list, which must contain the location of the IPA archive created in the Sign phase; and the IPA archive itself, the payload. If all these things are together on an accessible web server, a device whose UDID is included in the provisioning profile will be able to install the app. While you have created the IPA with this process you will probably create the html and plist manually and commit it to the source repository.
cURL5 is a great tool included with OS X. If you have ever worked with web services you may have used it to take a look at raw JSON or XML to figure out where interesting data is. What I didn’t know is that it can send data too. Sending files via ftp turned out to be simple. one of those rare trial-and-success experiences:
curl ${HOSTING_ADDRESS}${HOSTING_PATH} \
    -u ${HOSTING_NAME}:${HOSTING_PWD} \
    -Q "TYPE I" \
    -T ${WORKSPACE}/${TARGET_NAME}.ipa
Most of this should be very easy to understand. You have a host address and a path on that host where you want the html, plist and IPA to be placed. You know the name and password of an account on that server which has the required privileges to upload to it. You have the thing to send, specified by the -T argument – here the IPA is being transferred. A final note about the ftp protocol: it is ancient and assumes everything is 7-bit text unless you specify otherwise. If you transfer binaries without specifying “TYPE I” then the binary will become corrupted. For the html and plist assets this -Q argument can be omitted.
And you’re done! if all went well you should now be able to connect to your sever and install the app.

BUILD TRIGGERING

Hang on, this isn’t continuous integration yet, it only happens when you click “Schedule a build”. It should happen whenever code is pushed to the repository. This is easily done. The most easy way to do it is to poll the repository at intervals to see if anything changed. In the Build Triggers section of job configuration, Poll SCM allows you to set a schedule for this polling. Once every five minutes is specified by “*/5 * * * *”.
If you feel that polling in general is bad (and it is) and you are using GitHub, AND your build machine is accessible to GitHub (i.e. is on the public internet) then you can have GitHub notify your build machine when a job should be executed. Mine isn’t so I didn’t try this.6
Either way your final test is to push to your repository and wait to see if Jenkins starts a build. Make some change that you can see on your device, let it go through the whole process and download the result. Ah, satisfying.

NEXT…

This has been a very basic kind of how-to article. A lot more can be done to automate build, packaging and deployment and shell masters will find plenty of room to improve what I have shown here. It would be good practise to archive each deployed build. Multiple versions and configurations could be built at each build trigger. Automated testing could be run. Build lights7 could be set to show the result of each build. You could convert the VM to a physical machine to make use of a mini or similar as a build server.

THE SOURCE

Shell commands used inside Jenkins

HOSTING_NAME="[your account name on your deployment server]"
HOSTING_PWD="[your password on your deployment server]"
HOSTING_ADDRESS="ftp://ftp.[your server's address]/"
HOSTING_PATH="[path you will serve the build from]"
#
# interesting to know the path
#
echo ${WORKSPACE} 
#
# your provisioning profile can be called, and stored, as you like
#
TARGET_NAME="[Your target name]"
SIGNING_IDENTITY="iPhone Distribution: [Developer Name]"
PROVISIONING_PROFILE_DIR="/Users/jenkins/Library/MobileDevice/Provisioning Profiles"
PROVISIONING_PROFILE="[Target]_Ad_Hoc.mobileprovision"
#
# always set for xcodebuild
#
export DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer
#
# Lucky I found this on Stack Overflow. My build wasn't working until I did.
#
export CODESIGN_ALLOCATE="/Applications/Xcode.app/Contents/Developer/usr/bin/codesign_allocate"
#
# Unlock the keychain containing code signing keys and certificates
#
security unlock-keychain -p jenkins JenkinsCI
#
# Do the actual build
#
xcodebuild \
    -project [Project Name].xcodeproj \
    -target ${TARGET_NAME} \
    -configuration Release \
    -sdk iphoneos \
    CODE_SIGN_IDENTITY="${SIGNING_IDENTITY}"
#
# sign the .app so that it is ready for OTA
#
xcrun -sdk iphoneos PackageApplication \
    -o "${WORKSPACE}/${TARGET_NAME}.ipa" \
    -verbose "${WORKSPACE}/build/Release-iphoneos/${TARGET_NAME}.app" \
    -sign "${SIGNING_IDENTITY}" \
    --embed "${PROVISIONING_PROFILE_DIR}/${PROVISIONING_PROFILE}"
#
# place the new assets on the server so everyone can get them
#
# index.html
curl ${HOSTING_ADDRESS}${HOSTING_PATH} \
    -u ${HOSTING_NAME}:${HOSTING_PWD} \
    -T ${WORKSPACE}/Crawler/OTA/index.html
# plist
curl ${HOSTING_ADDRESS}${HOSTING_PATH} \
    -u ${HOSTING_NAME}:${HOSTING_PWD} \
    -T ${WORKSPACE}/Crawler/OTA/${TARGET_NAME}.plist
# IPA
curl ${HOSTING_ADDRESS}${HOSTING_PATH} \
    -u ${HOSTING_NAME}:${HOSTING_PWD} \
    -Q "TYPE I" \
    -T ${WORKSPACE}/${TARGET_NAME}.ipa

Sample (simple) html pointing to plist

<H3><a href="itms-services://?action=download-manifest&url=http://[server address].com/[path]/[Target name].plist">Download Target (Universal)</a></H3>

Property List identifying the location of the IPA

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
 <key>items</key>
 <array>
  <dict>
   <key>assets</key>
   <array>
    <dict>
     <key>kind</key>
     <string>software-package</string>
     <key>url</key>
     <string>http://www.[server address].com/[path]/[Target].ipa</string>
    </dict>
   </array>
   <key>metadata</key>
   <dict>
    <key>bundle-identifier</key>
    <string>[Bundle ID from your AppName-Info.plist file]</string>
    <key>bundle-version</key>
    <string>1.0</string>
    <key>kind</key>
    <string>software</string>
    <key>title</key>
    <string>[Target]</string>
   </dict>
  </dict>
 </array>
</dict>
</plist>
And that’s it – I hope you get something out of this post. Comments are disabled due to an infinitely high spam-to-actual-comment ratio in the past, but email is welcome. On twitter I’m @aeberbach, to email use the same name at iCloud.
1. Voltaire wrote in Candide, of Admiral Byng’s execution: “In this country, it is wise to kill an admiral from time to time to encourage the others.”
2. Apple Mac OS X 10.8.2 EULA – see section 2 (iii).
3. Apple WWDC 2012 Session 404, Building from the Command Line with Xcode – this is a very practical and useful session.