Re: [Akraino Blueprints] [Akraino Technical-Discuss] 答复: [Akraino CI Sub-Committee] Setting the nexus repo and storing the logs from CI Jenkins #gen

Andrew Wilkinson
 

The parent directory that is allocated by LF for your BP will be created when the first log is pushed.

 

Then you name any child directories you wish to use through the log push action

 

From: Yuyang (Gabriel) <gabriel.yuyang@...>
Sent: Tuesday, May 28, 2019 09:49
To: Andrew Wilkinson <andrew.wilkinson@...>; technical-discuss@...; ci@...; wutianwei <wutianwei1@...>; eball@...
Cc: KATHIRVEL, KANDAN <kk0563@...>; yroblamo@...
Subject: RE: [Akraino Blueprints] [Akraino Technical-Discuss] 答复: [Akraino CI Sub-Committee] Setting the nexus repo and storing the logs from CI Jenkins #gen

 

Hi Andrew,

 

NEXUS_PATH="${SILO}/job/${JOB_NAME}/${BUILD_NUMBER}"

Is this the R1 snapshot repo you mean? I suppose this repo will be created automatically when executing lftools deploy logs $NEXUS_URL $NEXUS_PATH $BUILD_URL

Is that correct!

 

Many thanks!

Gabriel

From: Andrew Wilkinson [mailto:andrew.wilkinson@...]
Sent: Wednesday, May 29, 2019 12:28 AM
To: Yuyang (Gabriel) <gabriel.yuyang@...>; technical-discuss@...; ci@...; wutianwei <wutianwei1@...>; eball@...
Cc: KATHIRVEL, KANDAN <kk0563@...>; yroblamo@...
Subject: RE: [Akraino Blueprints] [Akraino Technical-Discuss]
答复: [Akraino CI Sub-Committee] Setting the nexus repo and storing the logs from CI Jenkins #gen

 

No I don’t believe so (obviously you need to configure the Jenkins job specifically for your BP to do the CD on your HW and using the repos to be used for your R1 snapshot and consolidate any logs) but beyond that there’s nothing else I can think of

 

Andrew

 

From: Yuyang (Gabriel) <gabriel.yuyang@...>
Sent: Tuesday, May 28, 2019 09:23
To: Andrew Wilkinson <andrew.wilkinson@...>; technical-discuss@...; ci@...; wutianwei <wutianwei1@...>; eball@...
Cc: KATHIRVEL, KANDAN <kk0563@...>; yroblamo@...
Subject: RE: [Akraino Blueprints] [Akraino Technical-Discuss]
答复: [Akraino CI Sub-Committee] Setting the nexus repo and storing the logs from CI Jenkins #gen

 

Thanks Andrew!

 

Except user credentials of Nexus and lftools, are there additional configuration need to be done or tools need to be installed for a successful upload?

 

@Yolanda, could you help add something here?

 

 

Thanks in advance!

Gabriel

 

 

From: technical-discuss@... [mailto:technical-discuss@...] On Behalf Of Andrew Wilkinson
Sent: Tuesday, May 28, 2019 8:38 PM
To: technical-discuss@...; ci@...
Subject: Re: [Akraino Blueprints] [Akraino Technical-Discuss]
答复: [Akraino CI Sub-Committee] Setting the nexus repo and storing the logs from CI Jenkins #gen

 

Hi,

 

Recommend you use the LF tools created for the job executed within your Jenkins post build scripts https://pypi.org/project/lftools/  – an example from our Jenkins is shown below

 

 

 

# Deploying logs to LFNexus log server ##

# BUILD_NUMBER and JOB_NAME should be set by Jenkins

 

NEXUS_URL=https://nexus.akraino.org

SILO=ericsson

JENKINS_HOSTNAME=http://10.51.35.12/

JOB_NAME=Unicycle_OVS_DPDK_Deployment

BUILD_URL="${JENKINS_HOSTNAME}/job/${JOB_NAME}/${BUILD_NUMBER}/"

NEXUS_PATH="${SILO}/job/${JOB_NAME}/${BUILD_NUMBER}"

 

lftools deploy logs $NEXUS_URL $NEXUS_PATH $BUILD_URL

 

echo "Logs uploaded to $NEXUS_URL/content/sites/logs/$NEXUS_PATH"

 

Andrew

 

 

---------- Forwarded message ---------
From: Yolanda Robla Mota <yroblamo@...>
Date: Mon, May 27, 2019 at 10:26 PM
Subject: Re: [Akraino Blueprints] [Akraino Technical-Discuss]
答复: [Akraino CI Sub-Committee] Setting the nexus repo and storing the logs from CI Jenkins #gen
To: wutianwei <wutianwei1@...>
CC: Tina Tsou <tina.tsou@...>, Andrew Wilkinson <andrew.wilkinson@...>, cesar.berho@... <cesar.berho@...>, ci@... <ci@...>, blueprints@... <blueprints@...>, technical-discuss@... <technical-discuss@...>

 

Same here. I have a need to upload logs manually from an external jenkins server. What i was told is to request the creation of an user for Nexus. I sent the request, but still waiting for it to be created. So i'm being unable to upload the missing logs.

 

On Tue, May 28, 2019 at 4:29 AM wutianwei <wutianwei1@...> wrote:

Hi all,

 

Thanks for your reply.

I have checked the documents[1], it is about the logs server processes for Akraino Jenkins, but it doesn’t conduct how to configure peer Jenkins server to put the logs to the logs server.

I setup the peer Jenkins server in our lab.

Hi Andrew, Could you tell me how to configure the peer Jenkins to store the log to nexus repo?

 

[1] https://wiki.akraino.org/display/AK/Jenkins+Guide

 

Thanks

Tianwei Wu

 

 

 

发件人: ci@... [mailto:ci@...] 代表 Tina Tsou
发送时间: 2019528 0:50
收件人: Andrew Wilkinson <andrew.wilkinson@...>
抄送: cesar.berho@...; ci@...; blueprints@...; technical-discuss@...
: Re: [Akraino CI Sub-Committee] Setting the nexus repo and storing the logs from CI Jenkins #gen

 

Dear all,

 

Thanks for replying during National holiday.

 

For those BPs who haven’t done that, please do it in time.

 

Thank you,

Tina


On May 27, 2019, at 9:09 AM, Andrew Wilkinson <andrew.wilkinson@...> wrote:

We also posted detailed guides of how we did this for our BP

 

https://wiki.akraino.org/display/AK/Integration+of+Peer+Jenkins+with+LF+master+Jenkins

 

https://wiki.akraino.org/display/AK/Peer+Jenkins+Setup+Guide

 

Andrew

 

From: ci@... <ci@...> On Behalf Of Cesar Berho via Lists.Akraino.Org
Sent: Monday, May 27, 2019 09:07
To: Tina Tsou <tina.tsou@...>; ci@...; blueprints@...; technical-discuss@...; Andrew Wilkinson <andrew.wilkinson@...>
Cc: ci@...
Subject: Re: [Akraino CI Sub-Committee] Setting the nexus repo and storing the logs from CI Jenkins #gen

 

There is a section on the existing guide published at:

 

https://wiki.akraino.org/display/AK/Jenkins+Guide

 

That documents the log server process, sharing extract of the section below:

Log Server

While Jenkins stores the console logs on the Jenkins instance, this is short term and typically depending on the job type purged after a week. We highly recommend preferring the log server over the Jenkins system when viewing console logs as it reduces load on Jenkins and we compress logs stored on the log server so downloads are faster.

We store Log server archives for 6 months.

At the end of a build the job ships logs to a Nexus logs repo and can be conveniently accessed via the https://logs.akraino.org/ URL. The Job Build Description will contain the specific log server URL for a build log. Jobs triggered via Gerrit Trigger will have the URL to the logs left as a post build comment.

Example Jenkins Build Description:

Build logs: https://logs.akraino.org/releng/vex-yul-odl-jenkins-1/distribution-check-carbon/167

Example Gerrit Comment:

jenkins-releng                                                  03-05 16:24

Patch Set 6: Verified+1

Build Successful

https://jenkins.akraino.org/releng/job/builder-tox-verify-master/1066/ : SUCCESS

Logs: https://logs.akraino.org/releng/vex-yul-odl-jenkins-1/builder-tox-verify-master/1066

The log path pattern for the logs server is LOG_SERVER_URL/SILO/JENKINS_HOSTNAME/JOB_NAME/BUILD_NUMBER typically if you know the JOB_NAME and BUILD_NUMBER you can replace the paths before it to convert the URL between Jenkins and the Log server.

We compress and store individual log files in gzip (.gz) format on the Nexus log repository. You can access these files through the URL.

·         Jenkins Production: https://logs.akraino.org/production

·         Jenkins Sandbox: https://logs.akraino.org/sandbox

Log Cleanup Schedule

The log servers are setup with cron jobs that purge logs during regular scheduled intervals.

·         Jenkins Production: Delete logs everyday at 08:00 UTC which are older than 180 days.

·         Jenkins Sandbox: Delete logs and jobs every week on Saturday at 08:00 UTC.

Thanks,

Cesar

 

From: ci@... <ci@...> On Behalf Of Tina Tsou
Sent: Monday, May 27, 2019 10:56 AM
To: ci@...; blueprints@...; technical-discuss@...; andrew.wilkinson@...
Subject: [Akraino CI Sub-Committee] Setting the nexus repo and storing the logs from CI Jenkins #gen

 

Dear all,

 

Several BPs asked the questions how to set the nexus repo and storing the logs from CI Jenkins.

 

I see IEC and other blueprints have done that.

Can anyone document it somewhere in the wiki page?

 

 

Thank you,

Tina

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.



--

Yolanda Robla Mota

Principal Software Engineer, RHCE

Red Hat

C/Avellana 213

Urb Portugal

yroblamo@...    M: +34605641639    

Join technical-discuss@lists.akraino.org to automatically receive all group messages.