Some changes coming in 17.03...

  • 12 Replies
  • 266 Views
*

ron

  • Administrator
  • Guru
  • *****
  • 3,238
Some changes coming in 17.03...
« on: March 20, 2017, 08:07:48 »
As we continually seek improvements in 8th's performance, we sometimes stumble on impressive gains.

The "pool" mechanism which 8th has used in all versions until now, is going to change in 17.03 to one which is faster and uses less memory.  To wit:

All items in 8th (strings, arrays, etc.) are allocated out of pools of that kind of item.  Currently, the pools are pre-allocated with a fixed number of items, and if more than that number of items is required, another pool of that size is allocated.  When an item's refcount becomes zero, it is available (and any data it had is released).

The new mechanism does not pre-allocate items, but rather allocates them on-demand.  Items with a 0 refcount are put on a 'free list', which is first checked when a new item is required.  If there are no items on the free list, then an item is allocated.

This has some advantages over the old method.  First, the startup time is lessened because less memory is allocated up-front.  Second, allocation of new items is faster, especially in the normal case of items being used and then released.  The speed gains depend on the usage pattern, and range from 5 - 50% (yes, that's right).

A further advantage is that the pool mechanism is much simpler.  Not that you care that much...

What does it mean for you?

In the main, you won't care.  However:
  • The "pool size" command-line option will be removed
  • The "G:pool-size" word will be deprecated; it will be removed in another version or two
  • The "G:(stats)" word is changing to show total items allocated and number of items on the free list; the 'number of pools' is always '1'
  • Another word will be added to force deallocation of items on the free list (for a particular pool)

*

bugmagnet

  • Master
  • ****
  • 252
  • Software Engineer, Propelis P/L
Re: Some changes coming in 17.03...
« Reply #1 on: March 21, 2017, 06:43:48 »
Impressive! Good news all round.

Bruce/bugmagnet

*

RichAMead

  • beta
  • Guru
  • *****
  • 593
  • "We all live in a big black hole. No, really."
Re: Some changes coming in 17.03...
« Reply #2 on: March 23, 2017, 02:49:33 »
It would be great to have a way to request pre-allocation (onto the free list presumably) of a specified number of items - for those applications that have either an 'initial loading' of some ("large") number of items, or which otherwise are known to maintain a "large" number of some item type for the life of the app...

*

ron

  • Administrator
  • Guru
  • *****
  • 3,238
Re: Some changes coming in 17.03...
« Reply #3 on: March 23, 2017, 04:39:50 »
I suppose I could make "pool-size" pre-allocate N items and put them on the free list...

*

ron

  • Administrator
  • Guru
  • *****
  • 3,238
Re: Some changes coming in 17.03...
« Reply #4 on: March 24, 2017, 10:13:42 »
I just tested a version where 'pool-size' is a no-op vs. one where it pre-allocates N items.  There was no noticeable performance difference overall, despite trying to force bad results.

The reason there's no performance difference is that in the first case, the allocation cost is amortized over the entire run-time, while in the second case it happens up-front. 

It should not make any difference in practical terms in any event.  So 'pool-size' will be deprecated, since it will be a no-op

*

RichAMead

  • beta
  • Guru
  • *****
  • 593
  • "We all live in a big black hole. No, really."
Re: Some changes coming in 17.03...
« Reply #5 on: March 26, 2017, 05:59:09 »
It would matter if the creation of many objects at once occurs in the midst of running an application would it not - causing an unappealing 'pause' while the allocations occur?

*

ron

  • Administrator
  • Guru
  • *****
  • 3,238
Re: Some changes coming in 17.03...
« Reply #6 on: March 26, 2017, 06:20:01 »
It would cause a pause at that point; but a pause would occur somewhere else where you allocated the items up-front.

The same happens in any system where you need to allocate a ton of items.

*

RichAMead

  • beta
  • Guru
  • *****
  • 593
  • "We all live in a big black hole. No, really."
Re: Some changes coming in 17.03...
« Reply #7 on: March 26, 2017, 11:30:46 »
Sure, but it would generally be preferred up front rather than during (potential) user interaction, right?  Or in any case, preference would be to have some control over it?

On another note: I don't know how you've implemented your free list system but it may give you a nice performance boost if you were to assign new objects from those most recently freed (such as by putting the newly freed items at the head of the list rather than the tail if assigning new objects from the head) due to availability of those recent objects in on-core/on-chip/near-core cache (L1/L2 cache) rather than main memory...

*

ron

  • Administrator
  • Guru
  • *****
  • 3,238
Re: Some changes coming in 17.03...
« Reply #8 on: March 26, 2017, 11:33:19 »
OK, OK... I'll add a new word to pre-allocate items in a pool...

Yes, items from the free-list are LIFO so you get a nice boost from locality of reference.  Well, you do in some usage-patterns, anyway.

*

RichAMead

  • beta
  • Guru
  • *****
  • 593
  • "We all live in a big black hole. No, really."
Re: Some changes coming in 17.03...
« Reply #9 on: March 28, 2017, 00:37:52 »
Thanks - and awesome!

*

ron

  • Administrator
  • Guru
  • *****
  • 3,238
Re: Some changes coming in 17.03...
« Reply #10 on: March 28, 2017, 04:59:46 »
I would say, "Yes, I am"...  but that would be hubristic of me.

*

RichAMead

  • beta
  • Guru
  • *****
  • 593
  • "We all live in a big black hole. No, really."
Re: Some changes coming in 17.03...
« Reply #11 on: March 28, 2017, 16:49:25 »
 :o   ;D   8)   :P

*

ron

  • Administrator
  • Guru
  • *****
  • 3,238
Re: Some changes coming in 17.03...
« Reply #12 on: April 02, 2017, 07:14:01 »
Working on 17.03 for Thursday (06 Apr 2017) release. 

It will include the efficiency improvements as well as bug fixes, some new words, updated libraries and new samples.  If you've got any last-minute bugs you've found which haven't yet been reported, try to get them in by Tuesday.

Thanks!