Re: [Akraino TSC] New project presentation in Akraino Technical Call today: IEC Type 5: SmartNIC for Integrated Edge Cloud (IEC)
In answer to your point – sorry if it’s a bit long but it’s touching on a key aspect of Akraino so I think worth it.
>> That is going to add another dimension of complexity to the BPs that gets amplified with testing, etc... Are we sure we want to include them
directly in the BPs or just in the CI/testing later?
I think it is a choice of each BP PTL (with their team) as to what HW they decide to validate in their BP and submit to the community for approval (as either Core, Mature or Incubation self-certified).
For example today in our BP (the NC Unicycle OVS-DPDK BP) we already validate using a defined NIC – that’s what’s proven and deployed in our validation lab and it is with that NIC in Dell 740XD servers that we are requesting for inclusion as self certified in R2. [I would expect the BP to work on a HP DL380 but we haven’t validated that in the CD lab thus it’s not included as a validated configuration in the R2 release]
We haven’t validated any other config in R2 but we could, for example, in R3 choose to also validate against this smart NIC in the same or different server as well as the HW configuration we used in R1 and R2.
If we want to submit both configurations we would need to submit evidence of success for either (a) two deployments in the form of two logs from two different CDs, one with the original NICs and one with the SmartNICs or (b) we could submit a single CD with machines running both types of NICs if our BP supported that.
Either way we are submitting a proven successfully deployed HW+SW stack for TSC approval.
If we did do this I’d argue it’s the same BP but just a different HW configuration of a POD deployed by that BP.
But this is actually getting to a more fundamental point – from the technical charter: “A Blueprint is a declarative configuration of an edge Cloud Stack that includes a combination of infrastructure hardware components, SW components ….”
This DECLARATIVE set of specific components is what is validated when a CD is successful – the success (or failure) of that is shown by the CD logs loaded to LF Nexus after each CD in a validation lab. This is a cornerstone of the TSC’s approval of every BP in a release. This is one of the reasons we’ve reviewed every CD log and been very ‘specific’ that a BP’s R2 release documentation should only include that which has been proven to successfully deploy. A project can include other aspirational targets and capabilities but only that which is validated in an Akraino validation lab should be included in the release documentation for that BP
From: Tina Tsou <Tina.Tsou@...>
Sent: Thursday, December 5, 2019 14:40
Cc: Andrew Wilkinson <andrew.wilkinson@...>; technical-discuss@...
Subject: Re: [Akraino Technical-Discuss] [Akraino TSC] New project presentation in Akraino Technical Call today: IEC Type 5: SmartNIC for Integrated Edge Cloud (IEC)
Please refer to these sections in Akraino Technical Community Document.
18.104.22.168 Akraino Validation Projects
Multiple labs will be used to validate Akraino projects to ensure production quality of blueprint and/or feature project releases. The Community CI lab is owned by the Akraino community and LF. In addition a number of Edge Validation labs are owned by an Akraino community company, organization or participant.
Comprehensive validation of the blueprint within the community will include testing of the OS layer, the undercloud layer, the upper cloud layer, VNF layer and the application layer. Test results should be available to the Akraino Edge Stack community for review via automated push to the wiki. Any defects identified should be logged in JIRA and assigned to the responsible party. When defects are identified in upstream components, Akraino coordinators should be assigned defect ownership. Akraino coordinators will work with upstream communities to resolve defects. For defects that do not involve upstream communities, impacted Protect Technical Leaders should be assigned defect ownership.
Akraino Validation Projects will also include testing of Akraino releases.
An automation framework should be used for test procedures and leverage existing test automation suites for upstream projects as required. It is the responsibility of the blueprint submitter to ensure that the Edge Validation and Community CI labs can support comprehensive validation of the blueprint and cover all use case characteristics.
The TSC subcommittees tasked with the setup and structuring of operating procedures for the validation labs shall work to define the licensing requirements for the external labs.
22.214.171.124.1 Feature project unit testing in Community CI lab
The Akraino Community CI labs are owned by the Akraino community and LF. Access is controlled by LF best practices.
TSC will propose a subcommittee to maintain and coordinate the Community CI lab.
The following would be conducted in the Community CI lab:
Unit testing of Feature projects or
Integration of Feature projects to a blueprint
Upstream projects together in the support of Akraino Blueprints.
A full CD that works with the Akraino CI pipeline should be included in the validation project.
126.96.36.199.2 Blueprint and application testing in Akraino Edge Validation lab
Akraino Edge Validation labs are owned by an Akraino community company, organization or participant.
The Validation labs can be owned and operated by the supplier of the lab with limited access or no access to the hardware but results of the validation testing need to be shared to the community.
TSC will propose a subcommittee to maintain and coordinate the Validation labs.
Blueprint, application and VNF testing may be performed over a number of Validations labs concurrently in a coordinated manner as required.
The Validation labs should be able to connect to Akriano Edge Stack CI hosted by LF to pull in Akraino blueprints for the validation. Necessary Firewalls can be opened by the Akriano Community on request.
All Validation labs should be available most of the year for the community use and if the lab is not used for validation more than 3 consecutive months then it will be removed from the community list.
Hardware and Network configuration used within the Validation labs should be declared and information should be documented in the wiki
All the Validation labs addition, modifications or removable need to be approved by TSC
All historic results (minimum of 1 year) of blueprint validation, Applications and VNF testing should be maintained in the wiki.
188.8.131.52.2.1 Blueprint testing
The test plan and associated test cases to validate the blueprint will be published on the Akraino wiki and reviewed by blueprint stakeholders.
The lab configuration should support functional, performance, security, and other test cases for the blueprint and cover the full stack for the blueprint.
Any license or support beyond the Community supplied software should be funded, owned and operated by the Validation supplier
Open source test tools and vendor tools with appropriate configurations should drive test traffic profiles that mimic in-scope use cases. Akraino coordinators will engage upstream open source communities to get access to upstream community labs for the Akraino blueprint test team. The submitter is responsible to establish access to necessary vendor labs.
184.108.40.206.2.2 Application and VNF testing
Akraino blueprint releases ensure that applications and virtual network functions (VNFs) can on-board and operate effectively on the Akraino solution. Validation labs can be further used for the validation of Edge applications and VNFs.
Application and VNF testing may take a number of forms including performance, scalability, security and usability, resilience etc.
Submission of results to the Akriano community is optional in case of commercially sensitive applications.
Tina ^ ^
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.