[DISCUSS] Storage-class memory ecosystem program

classic Classic list List threaded Threaded
47 messages Options
123
Reply | Threaded
Open this post in threaded view
|

[DISCUSS] Storage-class memory ecosystem program

Gang(Gary) Wang
Hi all,

We can expect more and more projects will take the huge potential
advantages of storage-class memory for data processing and analytics
because silicon companies are able to produce high capacity non-volatile
memory on a large scale, this hardware technology will fundamentally change
the way to construct high performance applications similar to what happened
when replacing tape with disk technology since the 1980s. so if possible, I
advocate establishing an Apache working group to enhance the collaboration
and synergies mentioned by Patrick Stuedi for storage-class memory
technology-oriented projects.

Best.
Gary.
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Storage-class memory ecosystem program

Denis Mag
Hi Gary,

I’m ready to represent Apache Ignite [1] community within the group. Some of our community folks have already benchmarked Intel Optane (3D XP) and there are plans to integrate with this technology natively.

[1] https://ignite.apache.org


Denis


> On Oct 19, 2017, at 11:55 AM, Gang(Gary) Wang <[hidden email]> wrote:
>
> Hi all,
>
> We can expect more and more projects will take the huge potential
> advantages of storage-class memory for data processing and analytics
> because silicon companies are able to produce high capacity non-volatile
> memory on a large scale, this hardware technology will fundamentally change
> the way to construct high performance applications similar to what happened
> when replacing tape with disk technology since the 1980s. so if possible, I
> advocate establishing an Apache working group to enhance the collaboration
> and synergies mentioned by Patrick Stuedi for storage-class memory
> technology-oriented projects.
>
> Best.
> Gary.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Storage-class memory ecosystem program

Debojyoti Dutta-2
In reply to this post by Gang(Gary) Wang
This is a great idea. Storage class memory will have a big impact on many of the Apache projects.

Debo

Sent from my iPhone

> On Oct 19, 2017, at 11:55 AM, Gang(Gary) Wang <[hidden email]> wrote:
>
> Hi all,
>
> We can expect more and more projects will take the huge potential
> advantages of storage-class memory for data processing and analytics
> because silicon companies are able to produce high capacity non-volatile
> memory on a large scale, this hardware technology will fundamentally change
> the way to construct high performance applications similar to what happened
> when replacing tape with disk technology since the 1980s. so if possible, I
> advocate establishing an Apache working group to enhance the collaboration
> and synergies mentioned by Patrick Stuedi for storage-class memory
> technology-oriented projects.
>
> Best.
> Gary.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Storage-class memory ecosystem program

Wes McKinney
I'm happy to represent Apache Arrow on this. This is very much in line
with our focus on zero-copy / memory-mapped data access of structured
data sets and cache-efficient memory layout.

- Wes

On Thu, Oct 19, 2017 at 9:48 PM, Debojyoti Dutta <[hidden email]> wrote:

> This is a great idea. Storage class memory will have a big impact on many of the Apache projects.
>
> Debo
>
> Sent from my iPhone
>
>> On Oct 19, 2017, at 11:55 AM, Gang(Gary) Wang <[hidden email]> wrote:
>>
>> Hi all,
>>
>> We can expect more and more projects will take the huge potential
>> advantages of storage-class memory for data processing and analytics
>> because silicon companies are able to produce high capacity non-volatile
>> memory on a large scale, this hardware technology will fundamentally change
>> the way to construct high performance applications similar to what happened
>> when replacing tape with disk technology since the 1980s. so if possible, I
>> advocate establishing an Apache working group to enhance the collaboration
>> and synergies mentioned by Patrick Stuedi for storage-class memory
>> technology-oriented projects.
>>
>> Best.
>> Gary.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Storage-class memory ecosystem program

Anoop John
+1..    Am there to represent Apache HBase.

-Anoop-

On Fri, Oct 20, 2017 at 8:14 AM, Wes McKinney <[hidden email]> wrote:

> I'm happy to represent Apache Arrow on this. This is very much in line
> with our focus on zero-copy / memory-mapped data access of structured
> data sets and cache-efficient memory layout.
>
> - Wes
>
> On Thu, Oct 19, 2017 at 9:48 PM, Debojyoti Dutta <[hidden email]> wrote:
>> This is a great idea. Storage class memory will have a big impact on many of the Apache projects.
>>
>> Debo
>>
>> Sent from my iPhone
>>
>>> On Oct 19, 2017, at 11:55 AM, Gang(Gary) Wang <[hidden email]> wrote:
>>>
>>> Hi all,
>>>
>>> We can expect more and more projects will take the huge potential
>>> advantages of storage-class memory for data processing and analytics
>>> because silicon companies are able to produce high capacity non-volatile
>>> memory on a large scale, this hardware technology will fundamentally change
>>> the way to construct high performance applications similar to what happened
>>> when replacing tape with disk technology since the 1980s. so if possible, I
>>> advocate establishing an Apache working group to enhance the collaboration
>>> and synergies mentioned by Patrick Stuedi for storage-class memory
>>> technology-oriented projects.
>>>
>>> Best.
>>> Gary.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Storage-class memory ecosystem program

Johnu
Interested. I agree with Gary.  Great to see the collaboration effort.

On Thu, Oct 19, 2017 at 9:59 PM, Anoop John <[hidden email]> wrote:

> +1..    Am there to represent Apache HBase.
>
> -Anoop-
>
> On Fri, Oct 20, 2017 at 8:14 AM, Wes McKinney <[hidden email]> wrote:
>> I'm happy to represent Apache Arrow on this. This is very much in line
>> with our focus on zero-copy / memory-mapped data access of structured
>> data sets and cache-efficient memory layout.
>>
>> - Wes
>>
>> On Thu, Oct 19, 2017 at 9:48 PM, Debojyoti Dutta <[hidden email]> wrote:
>>> This is a great idea. Storage class memory will have a big impact on many of the Apache projects.
>>>
>>> Debo
>>>
>>> Sent from my iPhone
>>>
>>>> On Oct 19, 2017, at 11:55 AM, Gang(Gary) Wang <[hidden email]> wrote:
>>>>
>>>> Hi all,
>>>>
>>>> We can expect more and more projects will take the huge potential
>>>> advantages of storage-class memory for data processing and analytics
>>>> because silicon companies are able to produce high capacity non-volatile
>>>> memory on a large scale, this hardware technology will fundamentally change
>>>> the way to construct high performance applications similar to what happened
>>>> when replacing tape with disk technology since the 1980s. so if possible, I
>>>> advocate establishing an Apache working group to enhance the collaboration
>>>> and synergies mentioned by Patrick Stuedi for storage-class memory
>>>> technology-oriented projects.
>>>>
>>>> Best.
>>>> Gary.
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [hidden email]
>>> For additional commands, e-mail: [hidden email]
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Storage-class memory ecosystem program

Atri Sharma
In reply to this post by Anoop John
+1

On Fri, Oct 20, 2017 at 10:29 AM, Anoop John <[hidden email]> wrote:

> +1..    Am there to represent Apache HBase.
>
> -Anoop-
>
> On Fri, Oct 20, 2017 at 8:14 AM, Wes McKinney <[hidden email]> wrote:
>> I'm happy to represent Apache Arrow on this. This is very much in line
>> with our focus on zero-copy / memory-mapped data access of structured
>> data sets and cache-efficient memory layout.
>>
>> - Wes
>>
>> On Thu, Oct 19, 2017 at 9:48 PM, Debojyoti Dutta <[hidden email]> wrote:
>>> This is a great idea. Storage class memory will have a big impact on many of the Apache projects.
>>>
>>> Debo
>>>
>>> Sent from my iPhone
>>>
>>>> On Oct 19, 2017, at 11:55 AM, Gang(Gary) Wang <[hidden email]> wrote:
>>>>
>>>> Hi all,
>>>>
>>>> We can expect more and more projects will take the huge potential
>>>> advantages of storage-class memory for data processing and analytics
>>>> because silicon companies are able to produce high capacity non-volatile
>>>> memory on a large scale, this hardware technology will fundamentally change
>>>> the way to construct high performance applications similar to what happened
>>>> when replacing tape with disk technology since the 1980s. so if possible, I
>>>> advocate establishing an Apache working group to enhance the collaboration
>>>> and synergies mentioned by Patrick Stuedi for storage-class memory
>>>> technology-oriented projects.
>>>>
>>>> Best.
>>>> Gary.
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [hidden email]
>>> For additional commands, e-mail: [hidden email]
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>



--
Regards,

Atri
l'apprenant

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Storage-class memory ecosystem program

Gang(Gary) Wang
It is wonderful, as of now, we have the following projects and their
communities.

   - *Ignite* represented by Denis Magda
   - *Arrow *represented by Wes McKinney
   - *Hbase *represented by Anoop John
   - *Mnemonic *represented by* Gary*

With above projects, we can cover Storage-class memory oriented Database,
KV Store, Columnar Structured Dataset, Durable Object Model, Durable
Computing Model for new generation high-performance applications, e.g. data
querying, processing, and analytics.


On Thu, Oct 19, 2017 at 11:01 PM, Atri Sharma <[hidden email]> wrote:

> +1
>
> On Fri, Oct 20, 2017 at 10:29 AM, Anoop John <[hidden email]>
> wrote:
> > +1..    Am there to represent Apache HBase.
> >
> > -Anoop-
> >
> > On Fri, Oct 20, 2017 at 8:14 AM, Wes McKinney <[hidden email]>
> wrote:
> >> I'm happy to represent Apache Arrow on this. This is very much in line
> >> with our focus on zero-copy / memory-mapped data access of structured
> >> data sets and cache-efficient memory layout.
> >>
> >> - Wes
> >>
> >> On Thu, Oct 19, 2017 at 9:48 PM, Debojyoti Dutta <[hidden email]>
> wrote:
> >>> This is a great idea. Storage class memory will have a big impact on
> many of the Apache projects.
> >>>
> >>> Debo
> >>>
> >>> Sent from my iPhone
> >>>
> >>>> On Oct 19, 2017, at 11:55 AM, Gang(Gary) Wang <[hidden email]>
> wrote:
> >>>>
> >>>> Hi all,
> >>>>
> >>>> We can expect more and more projects will take the huge potential
> >>>> advantages of storage-class memory for data processing and analytics
> >>>> because silicon companies are able to produce high capacity
> non-volatile
> >>>> memory on a large scale, this hardware technology will fundamentally
> change
> >>>> the way to construct high performance applications similar to what
> happened
> >>>> when replacing tape with disk technology since the 1980s. so if
> possible, I
> >>>> advocate establishing an Apache working group to enhance the
> collaboration
> >>>> and synergies mentioned by Patrick Stuedi for storage-class memory
> >>>> technology-oriented projects.
> >>>>
> >>>> Best.
> >>>> Gary.
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: [hidden email]
> >>> For additional commands, e-mail: [hidden email]
> >>>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: [hidden email]
> >> For additional commands, e-mail: [hidden email]
> >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > For additional commands, e-mail: [hidden email]
> >
>
>
>
> --
> Regards,
>
> Atri
> l'apprenant
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Storage-class memory ecosystem program

patrick stuedi
I'd be happy to be involved in the discussions as well. This matches
well with the focus we have in the Crail project (www.crail.io) where
we expose new storage technologies, including storage class memory, to
distributed data processing frameworks over high-speed network
fabrics. We are currently working on applying for Apache incubator
with Crail and we're looking for additional mentors :-)

On Fri, Oct 20, 2017 at 6:54 PM, Gang(Gary) Wang <[hidden email]> wrote:

> It is wonderful, as of now, we have the following projects and their
> communities.
>
>    - *Ignite* represented by Denis Magda
>    - *Arrow *represented by Wes McKinney
>    - *Hbase *represented by Anoop John
>    - *Mnemonic *represented by* Gary*
>
> With above projects, we can cover Storage-class memory oriented Database,
> KV Store, Columnar Structured Dataset, Durable Object Model, Durable
> Computing Model for new generation high-performance applications, e.g. data
> querying, processing, and analytics.
>
>
> On Thu, Oct 19, 2017 at 11:01 PM, Atri Sharma <[hidden email]> wrote:
>
>> +1
>>
>> On Fri, Oct 20, 2017 at 10:29 AM, Anoop John <[hidden email]>
>> wrote:
>> > +1..    Am there to represent Apache HBase.
>> >
>> > -Anoop-
>> >
>> > On Fri, Oct 20, 2017 at 8:14 AM, Wes McKinney <[hidden email]>
>> wrote:
>> >> I'm happy to represent Apache Arrow on this. This is very much in line
>> >> with our focus on zero-copy / memory-mapped data access of structured
>> >> data sets and cache-efficient memory layout.
>> >>
>> >> - Wes
>> >>
>> >> On Thu, Oct 19, 2017 at 9:48 PM, Debojyoti Dutta <[hidden email]>
>> wrote:
>> >>> This is a great idea. Storage class memory will have a big impact on
>> many of the Apache projects.
>> >>>
>> >>> Debo
>> >>>
>> >>> Sent from my iPhone
>> >>>
>> >>>> On Oct 19, 2017, at 11:55 AM, Gang(Gary) Wang <[hidden email]>
>> wrote:
>> >>>>
>> >>>> Hi all,
>> >>>>
>> >>>> We can expect more and more projects will take the huge potential
>> >>>> advantages of storage-class memory for data processing and analytics
>> >>>> because silicon companies are able to produce high capacity
>> non-volatile
>> >>>> memory on a large scale, this hardware technology will fundamentally
>> change
>> >>>> the way to construct high performance applications similar to what
>> happened
>> >>>> when replacing tape with disk technology since the 1980s. so if
>> possible, I
>> >>>> advocate establishing an Apache working group to enhance the
>> collaboration
>> >>>> and synergies mentioned by Patrick Stuedi for storage-class memory
>> >>>> technology-oriented projects.
>> >>>>
>> >>>> Best.
>> >>>> Gary.
>> >>>
>> >>> ---------------------------------------------------------------------
>> >>> To unsubscribe, e-mail: [hidden email]
>> >>> For additional commands, e-mail: [hidden email]
>> >>>
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: [hidden email]
>> >> For additional commands, e-mail: [hidden email]
>> >>
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: [hidden email]
>> > For additional commands, e-mail: [hidden email]
>> >
>>
>>
>>
>> --
>> Regards,
>>
>> Atri
>> l'apprenant
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Storage-class memory ecosystem program

Debo Dutta (dedutta)
Hi Patrick

Am happy to help w Crail.

Thx
Debo

Sent from my iPhone

On Oct 20, 2017, at 10:14 AM, patrick stuedi <[hidden email]<mailto:[hidden email]>> wrote:

I'd be happy to be involved in the discussions as well. This matches
well with the focus we have in the Crail project (www.crail.io<http://www.crail.io>) where
we expose new storage technologies, including storage class memory, to
distributed data processing frameworks over high-speed network
fabrics. We are currently working on applying for Apache incubator
with Crail and we're looking for additional mentors :-)

On Fri, Oct 20, 2017 at 6:54 PM, Gang(Gary) Wang <[hidden email]<mailto:[hidden email]>> wrote:
It is wonderful, as of now, we have the following projects and their
communities.

  - *Ignite* represented by Denis Magda
  - *Arrow *represented by Wes McKinney
  - *Hbase *represented by Anoop John
  - *Mnemonic *represented by* Gary*

With above projects, we can cover Storage-class memory oriented Database,
KV Store, Columnar Structured Dataset, Durable Object Model, Durable
Computing Model for new generation high-performance applications, e.g. data
querying, processing, and analytics.


On Thu, Oct 19, 2017 at 11:01 PM, Atri Sharma <[hidden email]<mailto:[hidden email]>> wrote:

+1

On Fri, Oct 20, 2017 at 10:29 AM, Anoop John <[hidden email]<mailto:[hidden email]>>
wrote:
+1..    Am there to represent Apache HBase.

-Anoop-

On Fri, Oct 20, 2017 at 8:14 AM, Wes McKinney <[hidden email]<mailto:[hidden email]>>
wrote:
I'm happy to represent Apache Arrow on this. This is very much in line
with our focus on zero-copy / memory-mapped data access of structured
data sets and cache-efficient memory layout.

- Wes

On Thu, Oct 19, 2017 at 9:48 PM, Debojyoti Dutta <[hidden email]<mailto:[hidden email]>>
wrote:
This is a great idea. Storage class memory will have a big impact on
many of the Apache projects.

Debo

Sent from my iPhone

On Oct 19, 2017, at 11:55 AM, Gang(Gary) Wang <[hidden email]<mailto:[hidden email]>>
wrote:

Hi all,

We can expect more and more projects will take the huge potential
advantages of storage-class memory for data processing and analytics
because silicon companies are able to produce high capacity
non-volatile
memory on a large scale, this hardware technology will fundamentally
change
the way to construct high performance applications similar to what
happened
when replacing tape with disk technology since the 1980s. so if
possible, I
advocate establishing an Apache working group to enhance the
collaboration
and synergies mentioned by Patrick Stuedi for storage-class memory
technology-oriented projects.

Best.
Gary.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]<mailto:[hidden email]>
For additional commands, e-mail: [hidden email]<mailto:[hidden email]>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]<mailto:[hidden email]>
For additional commands, e-mail: [hidden email]<mailto:[hidden email]>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]<mailto:[hidden email]>
For additional commands, e-mail: [hidden email]<mailto:[hidden email]>




--
Regards,

Atri
l'apprenant

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]<mailto:[hidden email]>
For additional commands, e-mail: [hidden email]<mailto:[hidden email]>



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]<mailto:[hidden email]>
For additional commands, e-mail: [hidden email]<mailto:[hidden email]>

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Storage-class memory ecosystem program

Gary Wong
Add Crail

   - *Ignite* represented by Denis Magda
   - *Arrow *represented by Wes McKinney
   - *Hbase *represented by Anoop John
   - *Crail* represented by *Patrick Stuedi*
   - *Mnemonic *represented by* Gary*

With above projects, we can cover Storage-class memory oriented *Distributed
Database, KV Store, Columnar Structured Dataset, Distributed Data
Store, **Durable
Object Model, Durable Computing Model* for new generation high-performance
applications, e.g. data querying, processing, and analytics.

On Fri, Oct 20, 2017 at 10:28 AM, Debo Dutta (dedutta) <[hidden email]>
wrote:

> Hi Patrick
>
> Am happy to help w Crail.
>
> Thx
> Debo
>
> Sent from my iPhone
>
> On Oct 20, 2017, at 10:14 AM, patrick stuedi <[hidden email]<mailto:
> [hidden email]>> wrote:
>
> I'd be happy to be involved in the discussions as well. This matches
> well with the focus we have in the Crail project (www.crail.io<http://www.
> crail.io>) where
> we expose new storage technologies, including storage class memory, to
> distributed data processing frameworks over high-speed network
> fabrics. We are currently working on applying for Apache incubator
> with Crail and we're looking for additional mentors :-)
>
> On Fri, Oct 20, 2017 at 6:54 PM, Gang(Gary) Wang <[hidden email]<mailto:
> [hidden email]>> wrote:
> It is wonderful, as of now, we have the following projects and their
> communities.
>
>   - *Ignite* represented by Denis Magda
>   - *Arrow *represented by Wes McKinney
>   - *Hbase *represented by Anoop John
>   - *Mnemonic *represented by* Gary*
>
> With above projects, we can cover Storage-class memory oriented Database,
> KV Store, Columnar Structured Dataset, Durable Object Model, Durable
> Computing Model for new generation high-performance applications, e.g. data
> querying, processing, and analytics.
>
>
> On Thu, Oct 19, 2017 at 11:01 PM, Atri Sharma <[hidden email]<mailto:
> [hidden email]>> wrote:
>
> +1
>
> On Fri, Oct 20, 2017 at 10:29 AM, Anoop John <[hidden email]
> <mailto:[hidden email]>>
> wrote:
> +1..    Am there to represent Apache HBase.
>
> -Anoop-
>
> On Fri, Oct 20, 2017 at 8:14 AM, Wes McKinney <[hidden email]<mailto:
> [hidden email]>>
> wrote:
> I'm happy to represent Apache Arrow on this. This is very much in line
> with our focus on zero-copy / memory-mapped data access of structured
> data sets and cache-efficient memory layout.
>
> - Wes
>
> On Thu, Oct 19, 2017 at 9:48 PM, Debojyoti Dutta <[hidden email]<mailto:
> [hidden email]>>
> wrote:
> This is a great idea. Storage class memory will have a big impact on
> many of the Apache projects.
>
> Debo
>
> Sent from my iPhone
>
> On Oct 19, 2017, at 11:55 AM, Gang(Gary) Wang <[hidden email]<mailto:
> [hidden email]>>
> wrote:
>
> Hi all,
>
> We can expect more and more projects will take the huge potential
> advantages of storage-class memory for data processing and analytics
> because silicon companies are able to produce high capacity
> non-volatile
> memory on a large scale, this hardware technology will fundamentally
> change
> the way to construct high performance applications similar to what
> happened
> when replacing tape with disk technology since the 1980s. so if
> possible, I
> advocate establishing an Apache working group to enhance the
> collaboration
> and synergies mentioned by Patrick Stuedi for storage-class memory
> technology-oriented projects.
>
> Best.
> Gary.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]<mailto:
> [hidden email]>
> For additional commands, e-mail: [hidden email]<mailto:
> [hidden email]>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]<mailto:
> [hidden email]>
> For additional commands, e-mail: [hidden email]<mailto:
> [hidden email]>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]<mailto:
> [hidden email]>
> For additional commands, e-mail: [hidden email]<mailto:
> [hidden email]>
>
>
>
>
> --
> Regards,
>
> Atri
> l'apprenant
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]<mailto:
> [hidden email]>
> For additional commands, e-mail: [hidden email]<mailto:
> [hidden email]>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]<mailto:
> [hidden email]>
> For additional commands, e-mail: [hidden email]<mailto:
> [hidden email]>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Storage-class memory ecosystem program

Clebert Suconic
+1

On Sat, Oct 21, 2017 at 2:52 AM, Gary Wong <[hidden email]> wrote:

> Add Crail
>
>    - *Ignite* represented by Denis Magda
>    - *Arrow *represented by Wes McKinney
>    - *Hbase *represented by Anoop John
>    - *Crail* represented by *Patrick Stuedi*
>    - *Mnemonic *represented by* Gary*
>
> With above projects, we can cover Storage-class memory oriented *Distributed
> Database, KV Store, Columnar Structured Dataset, Distributed Data
> Store, **Durable
> Object Model, Durable Computing Model* for new generation high-performance
> applications, e.g. data querying, processing, and analytics.
>
> On Fri, Oct 20, 2017 at 10:28 AM, Debo Dutta (dedutta) <[hidden email]>
> wrote:
>
>> Hi Patrick
>>
>> Am happy to help w Crail.
>>
>> Thx
>> Debo
>>
>> Sent from my iPhone
>>
>> On Oct 20, 2017, at 10:14 AM, patrick stuedi <[hidden email]<mailto:
>> [hidden email]>> wrote:
>>
>> I'd be happy to be involved in the discussions as well. This matches
>> well with the focus we have in the Crail project (www.crail.io<http://www.
>> crail.io>) where
>> we expose new storage technologies, including storage class memory, to
>> distributed data processing frameworks over high-speed network
>> fabrics. We are currently working on applying for Apache incubator
>> with Crail and we're looking for additional mentors :-)
>>
>> On Fri, Oct 20, 2017 at 6:54 PM, Gang(Gary) Wang <[hidden email]<mailto:
>> [hidden email]>> wrote:
>> It is wonderful, as of now, we have the following projects and their
>> communities.
>>
>>   - *Ignite* represented by Denis Magda
>>   - *Arrow *represented by Wes McKinney
>>   - *Hbase *represented by Anoop John
>>   - *Mnemonic *represented by* Gary*
>>
>> With above projects, we can cover Storage-class memory oriented Database,
>> KV Store, Columnar Structured Dataset, Durable Object Model, Durable
>> Computing Model for new generation high-performance applications, e.g. data
>> querying, processing, and analytics.
>>
>>
>> On Thu, Oct 19, 2017 at 11:01 PM, Atri Sharma <[hidden email]<mailto:
>> [hidden email]>> wrote:
>>
>> +1
>>
>> On Fri, Oct 20, 2017 at 10:29 AM, Anoop John <[hidden email]
>> <mailto:[hidden email]>>
>> wrote:
>> +1..    Am there to represent Apache HBase.
>>
>> -Anoop-
>>
>> On Fri, Oct 20, 2017 at 8:14 AM, Wes McKinney <[hidden email]<mailto:
>> [hidden email]>>
>> wrote:
>> I'm happy to represent Apache Arrow on this. This is very much in line
>> with our focus on zero-copy / memory-mapped data access of structured
>> data sets and cache-efficient memory layout.
>>
>> - Wes
>>
>> On Thu, Oct 19, 2017 at 9:48 PM, Debojyoti Dutta <[hidden email]<mailto:
>> [hidden email]>>
>> wrote:
>> This is a great idea. Storage class memory will have a big impact on
>> many of the Apache projects.
>>
>> Debo
>>
>> Sent from my iPhone
>>
>> On Oct 19, 2017, at 11:55 AM, Gang(Gary) Wang <[hidden email]<mailto:
>> [hidden email]>>
>> wrote:
>>
>> Hi all,
>>
>> We can expect more and more projects will take the huge potential
>> advantages of storage-class memory for data processing and analytics
>> because silicon companies are able to produce high capacity
>> non-volatile
>> memory on a large scale, this hardware technology will fundamentally
>> change
>> the way to construct high performance applications similar to what
>> happened
>> when replacing tape with disk technology since the 1980s. so if
>> possible, I
>> advocate establishing an Apache working group to enhance the
>> collaboration
>> and synergies mentioned by Patrick Stuedi for storage-class memory
>> technology-oriented projects.
>>
>> Best.
>> Gary.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]<mailto:
>> [hidden email]>
>> For additional commands, e-mail: [hidden email]<mailto:
>> [hidden email]>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]<mailto:
>> [hidden email]>
>> For additional commands, e-mail: [hidden email]<mailto:
>> [hidden email]>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]<mailto:
>> [hidden email]>
>> For additional commands, e-mail: [hidden email]<mailto:
>> [hidden email]>
>>
>>
>>
>>
>> --
>> Regards,
>>
>> Atri
>> l'apprenant
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]<mailto:
>> [hidden email]>
>> For additional commands, e-mail: [hidden email]<mailto:
>> [hidden email]>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]<mailto:
>> [hidden email]>
>> For additional commands, e-mail: [hidden email]<mailto:
>> [hidden email]>
>>
>>



--
Clebert Suconic

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Storage-class memory ecosystem program

Gopal Vijayaraghavan
In reply to this post by Gang(Gary) Wang
> I'd be happy to be involved in the discussions as well. This matches
> well with the focus we have in the Crail project (www.crail.io) where
> we expose new storage technologies, including storage class memory, to
> distributed data processing frameworks over high-speed network
> fabrics. We are currently working on applying for Apache incubator
> with Crail and we're looking for additional mentors :-)

> On Fri, Oct 20, 2017 at 6:54 PM, Gang(Gary) Wang <[hidden email]> wrote:
>> It is wonderful, as of now, we have the following projects and their
>> communities.

I'd be happy to help this effort from Apache Hive/LLAP.

Hive/LLAP's cache has produced significant gains in NVDIMM experiments, integrating with a widely supported and common platform within Apache would be much better.

Cheers,
Gopal



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Storage-class memory ecosystem program

Owen O'Malley
In reply to this post by Gang(Gary) Wang
I can represent ORC within the group.

.. Owen

> On Oct 19, 2017, at 11:55 AM, Gang(Gary) Wang <[hidden email]> wrote:
>
> Hi all,
>
> We can expect more and more projects will take the huge potential
> advantages of storage-class memory for data processing and analytics
> because silicon companies are able to produce high capacity non-volatile
> memory on a large scale, this hardware technology will fundamentally change
> the way to construct high performance applications similar to what happened
> when replacing tape with disk technology since the 1980s. so if possible, I
> advocate establishing an Apache working group to enhance the collaboration
> and synergies mentioned by Patrick Stuedi for storage-class memory
> technology-oriented projects.
>
> Best.
> Gary.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Storage-class memory ecosystem program

Gang(Gary) Wang
Add ORC

   - *Ignite* represented by Denis Magda
   - *Arrow *represented by Wes McKinney
   - *Hbase *represented by Anoop John
   - *Crail* represented by *Patrick Stuedi*
   - *ORC *represented by* Owen O'Malley*
   - *Mnemonic *represented by* Gary*

With above projects, we could cover Storage-class memory oriented *Distributed
Database, KV Store, Columnar Structured Dataset, Distributed Data Store,
Columnar Storage, **Durable Object Model, Durable Computing Model* for new
generation high-performance applications, e.g. data querying, processing,
and analytics.


On Mon, Oct 23, 2017 at 1:21 PM, Owen O'Malley <[hidden email]>
wrote:

> I can represent ORC within the group.
>
> .. Owen
>
> > On Oct 19, 2017, at 11:55 AM, Gang(Gary) Wang <[hidden email]> wrote:
> >
> > Hi all,
> >
> > We can expect more and more projects will take the huge potential
> > advantages of storage-class memory for data processing and analytics
> > because silicon companies are able to produce high capacity non-volatile
> > memory on a large scale, this hardware technology will fundamentally
> change
> > the way to construct high performance applications similar to what
> happened
> > when replacing tape with disk technology since the 1980s. so if
> possible, I
> > advocate establishing an Apache working group to enhance the
> collaboration
> > and synergies mentioned by Patrick Stuedi for storage-class memory
> > technology-oriented projects.
> >
> > Best.
> > Gary.
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Storage-class memory ecosystem program

Gang(Gary) Wang
There are suggested initial goals for our workgroup

   - Sharing idea and good practice
   - Identifying common opportunities
   - Promoting storage-class memory application
   - Delivering solid solution
   - Avoiding reinvent the wheel
   - Integrating one another
   - Coordinating the progress


On Mon, Oct 23, 2017 at 4:02 PM, Gang(Gary) Wang <[hidden email]> wrote:

> Add ORC
>
>    - *Ignite* represented by Denis Magda
>    - *Arrow *represented by Wes McKinney
>    - *Hbase *represented by Anoop John
>    - *Crail* represented by *Patrick Stuedi*
>    - *ORC *represented by* Owen O'Malley*
>    - *Mnemonic *represented by* Gary*
>
> With above projects, we could cover Storage-class memory oriented *Distributed
> Database, KV Store, Columnar Structured Dataset, Distributed Data Store,
> Columnar Storage, **Durable Object Model, Durable Computing Model* for
> new generation high-performance applications, e.g. data querying,
> processing, and analytics.
>
>
> On Mon, Oct 23, 2017 at 1:21 PM, Owen O'Malley <[hidden email]>
> wrote:
>
>> I can represent ORC within the group.
>>
>> .. Owen
>>
>> > On Oct 19, 2017, at 11:55 AM, Gang(Gary) Wang <[hidden email]> wrote:
>> >
>> > Hi all,
>> >
>> > We can expect more and more projects will take the huge potential
>> > advantages of storage-class memory for data processing and analytics
>> > because silicon companies are able to produce high capacity non-volatile
>> > memory on a large scale, this hardware technology will fundamentally
>> change
>> > the way to construct high performance applications similar to what
>> happened
>> > when replacing tape with disk technology since the 1980s. so if
>> possible, I
>> > advocate establishing an Apache working group to enhance the
>> collaboration
>> > and synergies mentioned by Patrick Stuedi for storage-class memory
>> > technology-oriented projects.
>> >
>> > Best.
>> > Gary.
>>
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Storage-class memory ecosystem program

Debojyoti Dutta-2
Would love to help out in any way including working towards common benchmarks, APIs etc.

Debo

Sent from my iPhone

> On Oct 23, 2017, at 4:06 PM, Gang(Gary) Wang <[hidden email]> wrote:
>
> There are suggested initial goals for our workgroup
>
>   - Sharing idea and good practice
>   - Identifying common opportunities
>   - Promoting storage-class memory application
>   - Delivering solid solution
>   - Avoiding reinvent the wheel
>   - Integrating one another
>   - Coordinating the progress
>
>
>> On Mon, Oct 23, 2017 at 4:02 PM, Gang(Gary) Wang <[hidden email]> wrote:
>>
>> Add ORC
>>
>>   - *Ignite* represented by Denis Magda
>>   - *Arrow *represented by Wes McKinney
>>   - *Hbase *represented by Anoop John
>>   - *Crail* represented by *Patrick Stuedi*
>>   - *ORC *represented by* Owen O'Malley*
>>   - *Mnemonic *represented by* Gary*
>>
>> With above projects, we could cover Storage-class memory oriented *Distributed
>> Database, KV Store, Columnar Structured Dataset, Distributed Data Store,
>> Columnar Storage, **Durable Object Model, Durable Computing Model* for
>> new generation high-performance applications, e.g. data querying,
>> processing, and analytics.
>>
>>
>> On Mon, Oct 23, 2017 at 1:21 PM, Owen O'Malley <[hidden email]>
>> wrote:
>>
>>> I can represent ORC within the group.
>>>
>>> .. Owen
>>>
>>>> On Oct 19, 2017, at 11:55 AM, Gang(Gary) Wang <[hidden email]> wrote:
>>>>
>>>> Hi all,
>>>>
>>>> We can expect more and more projects will take the huge potential
>>>> advantages of storage-class memory for data processing and analytics
>>>> because silicon companies are able to produce high capacity non-volatile
>>>> memory on a large scale, this hardware technology will fundamentally
>>> change
>>>> the way to construct high performance applications similar to what
>>> happened
>>>> when replacing tape with disk technology since the 1980s. so if
>>> possible, I
>>>> advocate establishing an Apache working group to enhance the
>>> collaboration
>>>> and synergies mentioned by Patrick Stuedi for storage-class memory
>>>> technology-oriented projects.
>>>>
>>>> Best.
>>>> Gary.
>>>
>>>
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Storage-class memory ecosystem program

Von Gosling
In reply to this post by Gang(Gary) Wang
I'm happy to represent Apache RocketMQ on this  storage collaboration effort. This is a very key in low latency messaging.

Best Regards,
Von Gosling



> 在 2017年10月20日,02:55,Gang(Gary) Wang <[hidden email]> 写道:
>
> Hi all,
>
> We can expect more and more projects will take the huge potential
> advantages of storage-class memory for data processing and analytics
> because silicon companies are able to produce high capacity non-volatile
> memory on a large scale, this hardware technology will fundamentally change
> the way to construct high performance applications similar to what happened
> when replacing tape with disk technology since the 1980s. so if possible, I
> advocate establishing an Apache working group to enhance the collaboration
> and synergies mentioned by Patrick Stuedi for storage-class memory
> technology-oriented projects.
>
> Best.
> Gary.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Storage-class memory ecosystem program

Gang(Gary) Wang
Add *Hive/LLAP and RocketMQ*

   - *Ignite* represented by Denis Magda
   - *Arrow *represented by Wes McKinney
   - *Hbase *represented by Anoop John
   - *Crail* represented by *Patrick Stuedi*
   - *Hive/LLAP *represented by *Gopal Vijayaraghavan*
   - *RocketMQ  *represented by *Von Gosling*
   - *ORC *represented by* Owen O'Malley*
   - *Mnemonic *represented by* Gary*

With above projects, we could cover Storage-class memory oriented *Distributed
Database, KV Store, Columnar Structured Dataset, Distributed Data Store,
Columnar Storage, **Live Long And Process, Distributed Messaging and
Streaming, **Durable Object Model, Durable Computing Model* for new
generation high-performance applications, e.g. data querying, processing,
and analytics.

On Mon, Oct 23, 2017 at 9:32 PM, Von Gosling <[hidden email]> wrote:

> I'm happy to represent Apache RocketMQ on this  storage collaboration
> effort. This is a very key in low latency messaging.
>
> Best Regards,
> Von Gosling
>
>
>
> > 在 2017年10月20日,02:55,Gang(Gary) Wang <[hidden email]> 写道:
> >
> > Hi all,
> >
> > We can expect more and more projects will take the huge potential
> > advantages of storage-class memory for data processing and analytics
> > because silicon companies are able to produce high capacity non-volatile
> > memory on a large scale, this hardware technology will fundamentally
> change
> > the way to construct high performance applications similar to what
> happened
> > when replacing tape with disk technology since the 1980s. so if
> possible, I
> > advocate establishing an Apache working group to enhance the
> collaboration
> > and synergies mentioned by Patrick Stuedi for storage-class memory
> > technology-oriented projects.
> >
> > Best.
> > Gary.
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Storage-class memory ecosystem program

Gang(Gary) Wang
In reply to this post by Debojyoti Dutta-2
It is a great idea if we could have a common benchmark and APIs for
storage-class memory oriented library/framework/application, please go
ahead to propose one for discussion in our workgroup. Thanks!

On Mon, Oct 23, 2017 at 4:27 PM, Debojyoti Dutta <[hidden email]> wrote:

> Would love to help out in any way including working towards common
> benchmarks, APIs etc.
>
> Debo
>
> Sent from my iPhone
>
> > On Oct 23, 2017, at 4:06 PM, Gang(Gary) Wang <[hidden email]> wrote:
> >
> > There are suggested initial goals for our workgroup
> >
> >   - Sharing idea and good practice
> >   - Identifying common opportunities
> >   - Promoting storage-class memory application
> >   - Delivering solid solution
> >   - Avoiding reinvent the wheel
> >   - Integrating one another
> >   - Coordinating the progress
> >
> >
> >> On Mon, Oct 23, 2017 at 4:02 PM, Gang(Gary) Wang <[hidden email]>
> wrote:
> >>
> >> Add ORC
> >>
> >>   - *Ignite* represented by Denis Magda
> >>   - *Arrow *represented by Wes McKinney
> >>   - *Hbase *represented by Anoop John
> >>   - *Crail* represented by *Patrick Stuedi*
> >>   - *ORC *represented by* Owen O'Malley*
> >>   - *Mnemonic *represented by* Gary*
> >>
> >> With above projects, we could cover Storage-class memory oriented
> *Distributed
> >> Database, KV Store, Columnar Structured Dataset, Distributed Data Store,
> >> Columnar Storage, **Durable Object Model, Durable Computing Model* for
> >> new generation high-performance applications, e.g. data querying,
> >> processing, and analytics.
> >>
> >>
> >> On Mon, Oct 23, 2017 at 1:21 PM, Owen O'Malley <[hidden email]>
> >> wrote:
> >>
> >>> I can represent ORC within the group.
> >>>
> >>> .. Owen
> >>>
> >>>> On Oct 19, 2017, at 11:55 AM, Gang(Gary) Wang <[hidden email]>
> wrote:
> >>>>
> >>>> Hi all,
> >>>>
> >>>> We can expect more and more projects will take the huge potential
> >>>> advantages of storage-class memory for data processing and analytics
> >>>> because silicon companies are able to produce high capacity
> non-volatile
> >>>> memory on a large scale, this hardware technology will fundamentally
> >>> change
> >>>> the way to construct high performance applications similar to what
> >>> happened
> >>>> when replacing tape with disk technology since the 1980s. so if
> >>> possible, I
> >>>> advocate establishing an Apache working group to enhance the
> >>> collaboration
> >>>> and synergies mentioned by Patrick Stuedi for storage-class memory
> >>>> technology-oriented projects.
> >>>>
> >>>> Best.
> >>>> Gary.
> >>>
> >>>
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
123