AutomatingBuild

From Meta, a Wikimedia project coordination wiki
Jump to: navigation, search

Introduction[edit]

The scope of this wiki is to describe how RBCS build process is automated and steps involved in setting up the automation process. The wiki also specifies the pre-requisites required for setting up the build process

Pre-Requisite[edit]

Following softwares must be installed on your machine

  1. VS 2008 and Service pack2
  2. Python 2.6 - http://www.python.org/getit/
  3. _mssql/pymssql package(Python) - http://code.google.com/p/pymssql/downloads/list
  4. Active Perl - http://www.activestate.com/activeperl/downloads
  5. Tortoise SVN client - http://tortoisesvn.net/downloads.html
  6. Perforce client - http://www.perforce.com/perforce/downloads/platforms.html
  7. SQL server 2008 and SP2
  8. Putty - http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html
  9. Ruby - http://www.ruby-lang.org/en/downloads/

Environment Setup[edit]

1) Launch SVN client and go to following links and checkout/sync all the code onto your local machine
  1. https://ips.eng.symantec.com/svn/sigdev/trunk
  2. https://ips.eng.symantec.com/svn/sigdev/branches
  3. https://ips.eng.symantec.com/svn/automated/trunk/workflow_data
For more details in setting up SVN goto following details
  1. https://ips-response.corp.symantec.com/wiki/index.php/Subversion
2) Launch p4 client, provide proper credentials and server name as perforce.ges.symantec.com_1666 and checkout/sync from following repository
  1. //depot/Deepsight
  2. //depot/IDS_Response
For more details in setting up the perforce client go to following site:
  1. http://www.perforce.com/perforce/technical.html
  2. http://www.perforce.com/perforce/media_library/tutorials.html
3) Setup and verify the Signature Test framework. For more details please refer the following link:
  1. https://ips-response.corp.symantec.com/wiki/index.php/Response_Signature_Testing_Framework
4) Goto svn\sigdev\trunk\Projects\IPSQA\Tools\BuildTest folder and modify the path present in the following files to your machine specific path
  1. P4.bat
  2. P4Submit.bat
  3. QABuildCan.bat
  4. QABuildDeuce.bat
5) After this we will use windows scheduler to run the script. Please follow the below mentioned steps to add your python script to Windows scheduler
For Windows 7
  1. Launch task scheduler from Control Panel\All Control Panel Items\Administrative Tools and click on “Create Basic Task”
  2. Give the name of the task e.g. “Run Build” and Click Next
  3. In Trigger select the option “Daily”
  4. In Daily specify the exact time when we are suppose to create Build and give Recur every “1” day.
  5. Select start a program radio button
  6. Give the name of Python exe in Program/Script option
  7. In arguments specify the file name(CheckMetaData.py) which needs to be executed and click Next
  8. After that click finish
  9. In order to view your task in scheduler. Goto Task scheduler library in tree view. You can see your create scheduled task there
For Windows XP
  1. In Control panel goto Scheduled tasks and Click
  2. Go to File->New->Scheduled Tasks
  3. Rename the newly created task to “RunBuild” and Double click it
  4. In Task tab under run give the absolute path for Python.exe along with the file name(CheckMetaData.py)
  5. In Schedule tab specify Daily and exact time when you want to run the build
  6. Also provide “1” in Schedule task daily option in order to recur task daily
  7. Finally click on OK
  8. Your newly created task will be displayed under Scheduled tasks option
6) Goto svn\sigdev\trunk\Projects\IPSQA\Tools\BuildTest folder and change the path present in filePath.cfg as per the files present on your machine. The filePath.cfg contains path for following folder
  1. Release/Trunk
  2. For bat files (p4.bat, p4Sublim.bat, QaVuildDeuce.bat, QaBuildCanary.bat)
  3. Perforce directory path
  4. su-cr.md file
  5. Path for log files (for Canary and for Deuce)
  6. Path of a local folder where all the files after Deuce/Canary build will be copied
7) For creating build we need to run stored procedure present in the database. For setting up database connection we need to establish tunnel, please follow the steps below for setting up the tunnel
  1. Download and install the putty ssh client to the machine you are trying to connect from.
  2. Enter 10.160.20.251 for the host name
  3. Click Connection ->SSH -> Tunnels
  4. Fill out source port as 127.160.20.212:1433 and Destination as 10.160.20.212:1433. Click Add.
  5. Click Session -> Enter "RBCS QA" in box below Saved Sessions -> click Save.
  6. Click connect and enter the credentials root/d4s4n1 to log in. This session must remain connected and authenticated for the next 2 steps to work. Please do not give out these credentials!
  7. Open up SQL Server manager, and fill out the server name as 127.160.20.212. Select SQL Server Authentication and use your login credentials caval/rbcsrbcs. Click Connect.

Files in Build[edit]

Following is the list of files and their purpose while automating the build process
1. Filepath.cfg
It contains machine specific path of Release/trunk of SVN, path for various bat files e.g. p4.bat, p4 submit.bat, QaBuildDeuce.bat and QaBuildCanary.bat , path of local folder/files where the files will be copied after the build is completed for Deuce and Canary
2. P4.bat
Contains perforce command for sync and checkout /depot/IDS_Response/Dev_Work/chiraag_aval/SUIPSTest/ folder
3. P4Submit.bat
Contains perforce command for reverting and for submitting the changes
4. QABuildDeuce.bat
Contains commands for building\compiling Deuce signatures
5. QaBuildCanary.bat
Contains command for building\compiling Canary signatures
6. CheckMetaData.py
Python script for checking metadata. If metadata is old the script sleeps for 6 minutes. if metadata is updated it will execute BuildTest.py
7. BuildTest.py
Python script for creating and copying Deuce and Canary Build onto a local folder. After execution it will execute Downloadmetadata.py
8. DownloadMetaData.py
Python script for downloading metadata and running perforce commands for Sync with P4 and checkout, and for submitting the SU
9. PackageStatus.py
Python script used for executing stored procedure and for packaging build

User Flow[edit]