Jenkin Gerrit JJB

Материал из Wiki
Перейти к: навигация, поиск

Integration

Это часть лекции CI/CD за один день,

Before start development, we need to configure Jenkins.


Jenkins Job Builder (JJB) takes simple descriptions of Jenkins jobs in YAML or JSON format and uses them to configure Jenkins. You can keep your job descriptions in human readable text format in a version control system to make changes and auditing easier.
It also has a flexible template system, so creating many similarly configured jobs is easy.


For our Jenkins jobs we will provide the following workflow:

  • User creates JenkinsJob (yaml format)
  • User commit job into gerrit cicd-jenkins-job repo and send it to review
  • Jenkins trigger detects "review-created" event in gerrit and run syntax check job wich check yaml syntax
    • If job success: Voite +1
    • If job fails: Voite -1
  • In case of "+1" human can review and add +1
  • Merge
  • Jenkins trigger detects "merge" event in gerrit and run update job

As result, job defined in yaml and commited to repo will be tested and created in Jenkins.

Create repo in gerrit

Create project, which inherits all permission:
Jjb1.png

Install, configure and test JJB

Install

  • Install pip and libs:
apt-get install python-pip python-yaml
pip install  PyYAML python-jenkins
  • Install JJB
git clone https://git.openstack.org/openstack-infra/jenkins-job-builder
cd jenkins-job-builder/
python setup.py install
  • Now JJB is installed
# jenkins-jobs

usage: jenkins-jobs [-h] [--conf CONF] [-l LOG_LEVEL] [--ignore-cache]
                    [--flush-cache] [--version] [--allow-empty-variables]
                    {update,test,delete,delete-all} ...
jenkins-jobs: error: too few arguments

Configure JJB

JJB uses config file (defined with --conf option)
For jenkins I'd like to have separate user with separate key, so let's generate rsa key, create user in ldap and add this user to group

  • Generate key:
#su - jenkins

$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/var/lib/jenkins/.ssh/id_rsa):
Created directory '/var/lib/jenkins/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /var/lib/jenkins/.ssh/id_rsa.
Your public key has been saved in /var/lib/jenkins/.ssh/id_rsa.pub.
The key fingerprint is:
99:6c:fb:38:1e:5c:05:c9:ba:d6:cf:9b:f2:db:f0:a6 jenkins@cicd
The key's randomart image is:
+--[ RSA 2048]----+
|         ...     |
|          o.     |
|         .  .    |
|       ..o .     |
|        So.      |
|       ooo.      |
|       .+  o.    |
|        .+. o=.  |
|       .o..oE=o  |
+-----------------+
  • Create user in ldap (same password as for all other users):
    • Jenkins user:
dn: cn=Jenkins User,ou=cicd,dc=demo
changetype: add
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: inetOrgPerson
cn: Jenkins User
ou: cicd
uid: jenkins-user
givenName: Jenkins
sn: user1
userPassword: {SSHA}5oLdx/TJdGrRb3Jaz/9JWuFsj59pPoPt
mail: jenkins@jenkins.com
    • Add to group:
dn: cn=cicd  administrators,ou=cicd,dc=demo
changetype: modify
add: memberUid
memberUid: jenkins-user
    • Run commands:
# ldapmodify < jenkins-user
adding new entry "cn=Jenkins User,ou=cicd,dc=demo"
# ldapmodify < add_jenkins_to_group
modifying entry "cn=cicd  administrators,ou=cicd,dc=demo"
  • Check in LDAP


Jjb2.png


  • Check Login in Jenkins:


Jjb3.png


  • Log-in to Gerrit and add key


Jjb4.png


On this step w have user, which can manage jenkins, pull data from gerrit and vote for pathces

Test JJB

  • Create config jenkins_jobs.ini
[job_builder]
ignore_cache=True
keep_descriptions=False
include_path=.:scripts:~/git/
recursive=False
exclude=.*:manual:./development
allow_duplicates=False

[jenkins]
user=jenkins-user
password=r00tme
url=http://192.168.56.102:8080
query_plugins_info=True
##### This is deprecated, use job_builder section instead
#ignore_cache=True
  • Clone cicd-jenkins-job
$ git clone ssh://jenkins-user@192.168.56.102:29418/cicd-jenkins-job
Cloning into 'cicd-jenkins-job'...
remote: Counting objects: 2, done
remote: Finding sources: 100% (2/2)
remote: Total 2 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (2/2), done.
Checking connectivity... done.


  • Create test job doing nothing just show env
    • test.yaml
- job:
    name: 'test'
    project-type: freestyle
    description: |
        Test Job to test JJB dooing nothing
    concurrent: true
    wrappers:
      - timeout:
          timeout: 30
          fail: true
      - timestamps
      - ansicolor:
          colormap: xterm
    logrotate:
      daysToKeep: 1
      numToKeep: -1
      artifactDaysToKeep: -1
      artifactNumToKeep: -1
    properties:
      - heavy-job:
          weight: 1
    parameters:
      - string:
          name: PARAMETER1
          default: master
          description: |
             TEST_PARAMETER
    builders:
      - shell:
          !include-raw './builders/test.sh'
    publishers:
      - archive:
          allow-empty: true
          artifacts: 'artifacts/*.txt'
          latest-only: false


mkdir ./builders
  • ./builders/test.sh
#!/bin/bash



mkdir -p artifacts

env > artifacts/env.txt
  • Test syntax
jenkins-jobs --conf jenkins_jobs.ini test cicd-jenkins-job/demo/test.yaml
$ jenkins-jobs --conf jenkins_jobs.ini update  cicd-jenkins-job/demo/test.yaml

INFO:root:Updating jobs in ['cicd-jenkins-job/demo/test.yaml'] ([])
/usr/local/lib/python2.7/dist-packages/jenkins/__init__.py:530: DeprecationWarning: get_plugins_info() is deprecated, use get_plugins()
  DeprecationWarning)
WARNING:jenkins_jobs.local_yaml:tag '!include-raw' is deprecated, switch to using '!include-raw:'
INFO:jenkins_jobs.local_yaml:Including file './builders/test.sh' from path '/var/lib/jenkins/tmp/cicd-jenkins-job/demo'
WARNING:root:logrotate is deprecated on jenkins>=1.637, use the property build-discarder on newer jenkins instead
INFO:jenkins_jobs.builder:Number of jobs generated:  1
INFO:jenkins_jobs.builder:Creating jenkins job test
INFO:jenkins_jobs.builder:Cache saved
INFO:root:Number of jobs updated: 1
INFO:jenkins_jobs.builder:Cache saved


  • Check Jenkins:


Jjb5.png

Automate IT!

Next step is to automate jenkins update.

Configure Jenkins

  • Install Gerrit Trigger Plugin and Git Plugin


Jjb6.png

  • Restart Jenkins after plugin installation


Jjb7.png

We need to fix Gerrit Trigger Plugin configuration, it is designed for older versions of Geerrit

  • Go to trigger configuration


Jjb8.png

  • Add server:


Jjb9.png

  • No streaming right?

Go to the All-Projects->Access and under Global Capabilities add Stream Events to the LDAP Users group
Jjb10.png

  • Test it:
$ ssh  jenkins-user@192.168.56.102 -p 29418 gerrit stream-events
<empty stream!>



  • Set up +1 and -1 both for Verified and Code Review.
  • Go to 'Advanced Settings'


Jjb11.png

  • Option 'Verified' is deprecated now, so we need to fix plugin configuration.


Replace

 --verified <VERIFIED>

with :

--label 'Verified=<VERIFIED>' 




  • Configure user jenkins-user with ssh key, we need to access to repos


Jjb12.png

Create Test Job

  • Define job which should 'test job' before update it


File: test-jenkins-jobs.yaml

- job:
    name: 'test-jenkins-jobs'
    description: 'Test jenkins jobs configuration'
    concurrent: false
    node: 'master'
    builders:
      - shell:
          !include-raw: builders/test-jenkins-jobs/test-jenkins-jobs.sh
    properties:
      - heavy-job:
          weight: 1
    publishers:
      - postbuildscript:
          builders:
            - shell: 'rm jenkins_jobs.ini'
          script-only-if-succeeded: False
    parameters:
      - string:
          name: JJB_USER
          default: 'jenkins-user'
      - string:
          name: JJB_PASS
          default: 'r00tme'
      - string:
          name: JENKINS_URL
          default: 'http://192.168.56.102:8080'
    scm:
      - git:
          branches:
            - $GERRIT_REFSPEC
          remotes:
            - gerrit:
                refspec: $GERRIT_REFSPEC
                url: 'ssh://jenkins-user@192.168.56.102:29418/cicd-jenkins-job'
                credentials-id: 'cf14c3fb-2df7-45b6-9f8f-3f630e1f2c24'
          choosing-strategy: gerrit
          skip-tag: true
    triggers:
      - gerrit:
          trigger-on:
            - patchset-created-event:
                exclude-drafts: true
                exclude-trivial-rebase: true
                exclude-no-code-change: true
          projects:
            - project-compare-type: PLAIN
              project-pattern: 'cicd-jenkins-job'
              branches:
                - branch-compare-type: PLAIN
                  branch-pattern: 'master'
    wrappers:
      - timestamps
      - inject-passwords:
          global: true
  • Pay your attention on

credentials-id: '37227586-df68-4882-906c-52e3691f7eb2' , it is link to existing credential and differs from on installation to another.

Define builder script

mkdir test-jenkins-jobs
cat test-jenkins-jobs.sh

#!/bin/bash

set -ex


#source /home/jenkins/jjb/bin/activate

cat > jenkins_jobs.ini << EOF
[jenkins]
user=${JJB_USER}
password=${JJB_PASS}
url=${JENKINS_URL}
[job_builder]
ignore_cache=True
recursive=True
EOF

for D in `find .  -maxdepth 1 -type d   ! -path './.git*' ! -path '.'`;
do
  jenkins-jobs --conf jenkins_jobs.ini test ${D}  ${JOBS_LIST}
done
$ git add -A
$ git status
On branch master
Your branch is up-to-date with 'origin/master'.

Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

	new file:   builders/test-jenkins-jobs/test-jenkins-jobs.sh
	new file:   builders/test.sh
	new file:   test-jenkins-jobs.yaml
	new file:   test.yaml
  • Create .gitreview file
[gerrit]
host=192.168.56.102
port=29418
project= cicd-jenkins-job


# git review
remote: Processing changes: new: 1, refs: 1, done
remote:
remote: New Changes:
remote:   http://192.168.56.102:8081/r/6 test jobs
remote:
To ssh://mmaxur@192.168.56.102:29418/cicd-jenkins-job
 * [new branch]      HEAD -> refs/publish/master
  • Push job to Jenkins Server
# jenkins-jobs --conf jenkins_jobs.ini update  cicd-jenkins-job/demo/test-jenkins-jobs.yaml
ssh  jenkins-user@192.168.56.102 -p 29418 gerrit stream-events






{"uploader":{"name":"Max Mazur","email":"mmaxur@mirantis.com","username":"mmaxur"},"patchSet":{"number":"1","revision":"141ba5020057cc0827f16788ee5a53f3df8faa59","parents":["6e67c6fdac4410088cc5e80a896f474275e52f87"],"ref":"refs/changes/08/8/1","uploader":{"name":"Max Mazur","email":"mmaxur@mirantis.com","username":"mmaxur"},"createdOn":1458875515,"author":{"name":"Max Mazur","email":"mmaxur@mirantis.com","username":"mmaxur"},"isDraft":false,"kind":"REWORK","sizeInsertions":1,"sizeDeletions":0},"change":{"project":"cicd-jenkins-job","branch":"master","id":"Ie7a6cdf7b942c7378b74e65380fab818516ad0fb","number":"8","subject":"test commit 1","owner":{"name":"Max Mazur","email":"mmaxur@mirantis.com","username":"mmaxur"},"url":"http://192.168.56.102:8081/r/8","commitMessage":"test commit 1\n\nChange-Id: Ie7a6cdf7b942c7378b74e65380fab818516ad0fb\n","status":"NEW"},"type":"patchset-created","eventCreatedOn":1458875516}

  • Check created job:


Jjb13.png


  • Job Results:


Jjb15.png

Create Update Job

Flow it the same as for test job

Configure

  • Create YAML: update-jenkins-jobs.yaml
- job:
    name: 'update-jenkins-jobs'
    description: 'Update jenkins jobs configuration on jenkins-jobs merge or manually'
    concurrent: true
    node: 'master'
    builders:
      - shell:
          !include-raw: builders/update-jenkins-jobs/update-jenkins-jobs.sh
    parameters:
      - string:
          name: GERRIT_REFSPEC
          default: 'refs/heads/master'
      - string:
          name: JOBS_LIST
          description: 'Space separated list of jobs to update. Will update all jobs if empty'
      - string:
          name: JJB_USER
          default: 'jenkins-user'
      - string:
          name: JJB_PASS
          default: 'r00tme'
      - string:
          name: JENKINS_URL
          default: 'http://192.168.56.102:8080'
    properties:
      - heavy-job:
          weight: 1
    publishers:
      - postbuildscript:
          builders:
            - shell: 'rm jenkins_jobs.ini'
          script-only-if-succeeded: False
    scm:
      - git:
          branches:
            - $GERRIT_REFSPEC
          remotes:
            - gerrit:
                refspec: $GERRIT_REFSPEC
                url: 'ssh://jenkins-demo@192.168.56.102:29418/cicd-jenkins-job'
                credentials-id: 'cf14c3fb-2df7-45b6-9f8f-3f630e1f2c24'
          choosing-strategy: gerrit
          skip-tag: true
    triggers:
      - gerrit:
          trigger-on:
            - change-merged-event
          projects:
            - project-compare-type: PLAIN
              project-pattern: 'cicd-jenkins-job'
              branches:
                - branch-compare-type: PLAIN
                  branch-pattern: 'master'
    wrappers:
      - timestamps
      - inject-passwords:
          global: true


Create builder: update-jenkins-jobs/update-jenkins-jobs.sh

#!/bin/bash
set -ex



cat > jenkins_jobs.ini << EOF
[jenkins]
user=${JJB_USER}
password=${JJB_PASS}
url=${JENKINS_URL}
[job_builder]
ignore_cache=True
recursive=True
EOF

for D in `find .  -maxdepth 1 -type d   ! -path './.git*' ! -path '.'`;
do
  jenkins-jobs --conf jenkins_jobs.ini update ${D}
done


Add 'Update' job to Jenkins:

jenkins-jobs --conf jenkins_jobs.ini update  cicd-jenkins-job/demo/update-jenkins-jobs.yaml

Test

  • Test update job
    • Create test commit
    • See how test syntax job works (should set +1)
    • Merge
    • See how test.sh builder updated


  • Change, commit, send to review
# git add test.sh

# git commit -m 'test update job'
[master c80dc6a] test update job
 1 file changed, 1 insertion(+), 1 deletion(-)

# git review
remote: Processing changes: new: 1, refs: 1, done
remote:
remote: New Changes:
remote:   http://192.168.56.102:8081/r/10 test update job
remote:
To ssh://mmaxur@192.168.56.102:29418/cicd-jenkins-job
 * [new branch]      HEAD -> refs/publish/master


  • See created review


Jjb20.png

  • Merge it!


Jjb21.png

  • Update job was triggered, see console log.


Jjb22.png

  • Check changes in gerrit.


Jjb25.png


  • See, job is updated, same changes as committed.


Jjb23.png