Re: Akraino Charter

fzdarsky@...
 

On Mon, Sep 10, 2018 at 5:03 PM Srini <srinivasa.r.addepalli@...> wrote:

Hi Frank,

 

On this:

 

-        Is Akraino look to build the images/containers? Or limit the scope to installers, starting with Airship? If so, which sister project(s) going to build binary images suitable for Airship?

Could you maybe elaborate why you feel it matters whether building images is part of the project or not?

 

I see some projects applying patches (patches that are created in upstream) to their current release instead of taking latest release. This requires capability of building binaries & packages.


So selectively backporting patches from upstream's master into upstream's stable branch. That's something we might want to do in the upstream branch, too, but I can see the need for exceptions to that.
Hope the project can agree on a policy around such selective backporting (vs. carrying out-of-tree patches).
 

 

Even otherwise, there are some upstream projects don’t create binary images or images in a way that is required.

 

For example, Openstack-kolla and Openstack-helm projects can be considered as sister projects to Airship as Kolla project creates container images for each openstack service and pushes them to dockerhub.  Openstak-helm project is the home for helm charts which help in deploying them. I guess Airship installer requires them in containers and helm charts.  

 

Some projects such as libvirtd, ovs, ovs-dpdk, collectd, kubelet, OVN north/south controllers and many more, that are required to be part of the Edge stack, also need to be containerized and have helm charts. Where do we do them? As Akraino or as a sister project?

 

My thinking is that to make Akraino complete integration project, it would need to have capabilities to build the images. 


Agree there may be a need to build images. Should probably be left to the respective Blueprint projects to decide how to build them, as each project may have tooling that's more "native" to them.
 

 

Thoughts?

 

 

 

 

 

What about

 

 

From: main@... [mailto:main@...] On Behalf Of fzdarsky@...
Sent: Friday, September 7, 2018 4:20 AM
To: main@...
Subject: Re: [akraino] Akraino Charter

 

On Fri, Sep 7, 2018 at 12:02 AM Srini <srinivasa.r.addepalli@...> wrote:

Hi,

 

Few more thoughts. I guess we are all trying to see the scope of AkrainoJ.

 

It is my mental picture that integration project involves not only deployment, but also building images. 

But as of now, based on the gerrit repositories in Akraino, it seems that it seems to be deployment project and expects the images are built elsewhere.

The current code is only the seed code that AT&T generously donated to kick-start the project. Obviously there are pieces missing and maybe pieces we may not need / want to do differently later. I wouldn't extrapolate from the seed code to the project's mission/focus that the community needs to agree on.

 

Then, notice the code is integration code. It pulls in pieces from Airship and other upstreams (which themselves are development projects).

 

 

OPNFV is complete integration (seems like J) project and it works with various types of upstream projects.

 

-        Upstream projects that deliver binaries in various forms (example: Linux images)

o   RPMs, debian, docker containers etc…

-        Upstream projects that don’t build (needed) binaries : In this case,  various OPNFV projects (many, for example Barometer project) has ability to get source code (via git normally), build and make the images in OPNFV repositories.

-        Upstream projects with patch projects:  In this case, OPNFV projects (e.g Stor4NFV) has ability to get the source code, patch files from various git repos, patch the code and then build them to make binaries.

-        Upstream projects with some missing functionality:  Some OPNFV projects implemented the missing functionality. This functionality is patched/integrated with upstream projects to create binary images/containers.  For example, Barometer project has some collectd plugins, which are not part of upstream collected repos.

Ok, if you're not talking about patches for backporting but to fix/add functionality this would be forking and I hope we'll establish a clear upstream-first policy to prevent forking. Apart from that it would not be good citizenship to not upstream / not give upstream the chance to address gaps first, we'd accumulate technical debt that we don't want. I often hear the argument that these patches are "temporary" to enable us to move fast and that there's the intention to eventually upstream... which from experience hardly ever happens later.

 

It's a different, of course, if we're talking about plugins like in your Barometer example.

 

OPNFV CI/CD life cycle seems to be complete. In addition to building the images/containers, it also can invoke installers (deployment tools) to deploy the scenarios (images & Day-0 configuration) on Labs and then verify the end-to-end functionality. It is true that these are not production quality as the latest upstream projects may not have been tested thoroughly. Also, the number of test cases may not be sufficient to call it as production quality.

 

Questions for scope definition:

-        Is Akraino look to build the images/containers? Or limit the scope to installers, starting with Airship? If so, which sister project(s) going to build binary images suitable for Airship?

Could you maybe elaborate why you feel it matters whether building images is part of the project or not?

 

In my view, we should eventually build images to make internal testing and test-driving by users easier. But I'd like to avoid the trap of putting too much emphasis on the images; how they are built, how to harden them, how to tune/optimize them, etc. Just mentioning this, as it's a topic that typically comes up sooner or later (like recently in ONAP). And it's important to understand that users will likely throw away our images and rebuild & re-test anyway.

-        Is Akraino place to create the patches/enhancements/gaps? There are various opinions and it seems that many like to keep the scope of Akraino simple to make it successful.  But, one needs to think about sister projects where these gaps can be filled.

I'd expect that we'll identify gaps as a result of the integration. If it's a gap in an upstream project, we should absolutely strive to address those gaps there and never carry patches against an upstream project. If it's a functional gap not addressed by our current upstreams, we should try to find projects that do something similar and see whether we can extend those. Developing ourselves should be last resort and then as an independent (sub-)project that will need to prove its value to the larger community over time. Plus it should be possible to swap it out for other solutions if they develop elsewhere.

-        How does Akraino intend to make the deployment production quality if it does not maintain/create patches for upstream projects?

Whatever code we produce ourselves should of course strive towards production quality. But it's not our task to create patches to fix upstream projects or even do backports to an upstream's stable branch! That's the job of the upstream projects themselves.

 

And because everyone, both upstreams and us, has finite engineering resources, they have to trade off how much time they spend on backports to older, stable branches vs how much time they invest into new features. Plus how many HW/SW configurations they are able to fully test.

 

Now consider an edge stack that consists of dozens of components. It's already difficult to find combinations of versions of components that play well together, let alone the complexity of doing backports across all of them.

 

Next, consider how users would consume Akraino: Would they just download our/upstream images and run them in production? Of course not! They'd have to get commercial support from vendors (unless they have many engineers to burn to go DIY), but vendors will in most cases not support the exact combination as integrated/tested in Akraino, so the effort we put in there has a relatively low ROI...

 

TBH, this all makes the goal of "production quality" for the whole Akraino stack rather aspirational than realistic...

 

Based on the answers to above, we should keep the main scope of Akraino either as

 

-        Integration project

-        Deployment project (via Airship and others such as compass/apex in future)

-        Integration project + feature project.

 

 

Thanks

Srini

 

 

 

 

 

-----Original Message-----
From: main@... [mailto:main@...] On Behalf Of ildiko@...
Sent: Thursday, September 6, 2018 9:00 AM
To: main@...
Subject: Re: [akraino] Akraino Charter

 

I used mid-stream following the example of OPNFV without intentions on going into details on history.

 

I asked the question just to see where we stand today and to get a better picture on the intentions and expectations when it comes to contributing to and collaborating with Akraino. I assume the TSC is the decision maker on this one?

 

I believe that setting up the fundamentals right is very important and give clear definitions to people who will join later before we deep dive into implementation details. In that sense I have the same question/request about defining what a blueprint is which is still under discussion as far as I understand, so I will follow/participate in the community calls and discussions to figure that part out.

 

Thanks,

Ildikó

 

 

> On 2018. Sep 6., at 10:36, Margaret Chiosi <margaret.chiosi1@...> wrote:

>

> This may initially be an integration project like OPNFV – but not clear long term this is what we all agree on. I feel like OPNFV debate all over again.

> The reason why OPNFV turned into integration is because we could NOT attract enough developers.

> If we have the same challenge here (we need to be honest with ourselves) – then we should just then realize it will only be integration. If so, then we should take our learning on OPNFV to build akraino charter, labs, etc.

> Or maybe just build on OPNFV labs.

> Thank You,

> Margaret Chiosi

> VP Open Ecosystem Team

> Admin: Sophie Johnson

> Sophie.johnson1@...

> +1 (908) 541-3590

> Futurewei Technologies, Inc.

> Fixed Network Solution CC

> 400 Crossing Blvd

> Bridgewater, NJ 08807

> (cell) +1-732-216-5507

> <image001.png>

> From: main@... [mailto:main@...] On Behalf Of fzdarsky@...

> Sent: Thursday, September 06, 2018 9:33 AM

> To: main@...

> Subject: Re: [akraino] Akraino Charter

> It is confusing, yes. And it confirms that we've not made explicit whether Akraino is mainly an integration, development, or specification/standardization project.

> I hope we can agree that Akraino is an *integration project*, whose mission it is (paraphrasing from Charter) to make it easier for our users to design/customize, build, and operate edge stacks for their given use case.

> And no, this does not exclude us doing software development, e.g. testing tools&frameworks, adding auxiliary functionality for which no suitable upstream exists yet, etc.

> And no, this does not exclude us doing specifications, e.g. of Edge APIs where no suitable APIs exist yet.

> All it says is that such software development and specification work would be subordinate to the goal of enabling integration.

> Secondly, I hope we can further agree that Akraino's goal is to *enable* integration, *not prescribe* a given integration.

> Means, if our goal is to enable Akraino users to build an edge stack for Industrial IoT, Network Access Edge, etc., we'll ensure that the upstream components have the required functionality for those use cases and that this functionality also works end-to-end to meet those use cases, e.g. CPU resource management or GPU/TPU device management across app, middleware, Kubernetes and OpenStack. Or cross-stack operations.

> We are (I hope) not trying to create "Akraino distributions" and spend large amounts of engineering resources on redoing integration, stabilization, hardening, etc. work that's already done by upstream projects and downstream products.

> In that sense, I kind of disagree with calling Akraino a "mid-stream" project, because that would imply an expectation that downstream products that take source from upstreams like Kubernetes *via* Akraino instead of directly from them... which only makes sense if we envision to fork the upstreams or extend them in ways not acceptable to the upstreams...

> Thanks,

> Frank

> On Wed, Sep 5, 2018 at 10:15 PM <ildiko@...> wrote:

> Hi,

>

> I got a little confused on one part of the description below. What does “is also a home for Edge related open source projects for the functionality where there is a gap in the open source community” mean exactly?

>

> Is Akraino a mid-stream project meaning that beyond integration and testing it will also identify gaps and work together with other communities to address them in the open source projects? Or saying it is the home for those gaps means that it would be Akraino hosting the code for that missing functionality?

>

> Thanks,

> Ildikó

>

>

> > On 2018. Sep 4., at 11:49, Srini <srinivasa.r.addepalli@...> wrote:

> >

> > Hi Akraino TSC,

> >

> > In the last developer conference, TSC has taken an AI to come out with the Akraino charter definition.

> > Much of the discussion on last developer conference is mostly about blueprints, except for one break-out session.

> >

> > In regional orchestration, Edge API and AI breakout session, we talked about the need for them and the consensus is that Akraino project is the right place to develop them.  In one of the Kandan’s presentation, it was clear that Akraino is not only for developing blueprints for various use cases, but is also a home for Edge related open source projects for the functionality where there is a gap in the open source community. Please consider making the Akraino charter clear on the aspect of development projects to avoid any confusion.

> >

> > Thanks

> > Srini

> >

> >

> >

> >

>

>

>

>

>

> --

> Frank Zdarsky | NFV&SDN Technology Strategy, Office of the CTO | Red Hat

> e: fzdarsky@... | irc: fzdarsky@freenode | m: +49 175 82 11 64 4

>

 

 

 


 

--

Frank Zdarsky | NFV&SDN Technology Strategy, Office of the CTO | Red Hat
e: fzdarsky@... | irc: fzdarsky@freenode | m: +49 175 82 11 64 4



--
Frank Zdarsky | NFV&SDN Technology Strategy, Office of the CTO | Red Hat
e: fzdarsky@... | irc: fzdarsky@freenode | m: +49 175 82 11 64 4

Join main@lists.akraino.org to automatically receive all group messages.