Implement sched_[gs]etaffinity()

[Mark Geisert]

On Fri, 12 Apr 2019, Corinna Vinschen wrote:
> On Apr 11 13:52, Mark Geisert wrote:
>> On Thu, 11 Apr 2019, Corinna Vinschen wrote:
>>> On Apr 11 10:25, Corinna Vinschen wrote:
[...]
>>>> Your patch is nicely done, but what about machines with more than 64
>>>> CPUs?  Your patch only uses the standard API for up to 64 CPUs, so a
>>>> process can never use more than 64 CPUs or use CPUs from different CPU
>>>> groups.  There was also the case of this weird machine Achim Gratz once
>>>> worked on, which had less than 64 CPUs but *still* used multiple CPU
>>>> groups under Windows, for some reason.
>>>>
>>>> Any chance you could update your patch to support this functionality?
>>>> For some info, see MSDN:
>>>>
>>>> https://docs.microsoft.com/en-us/windows/desktop/ProcThread/processor-groups
>>>>
>>>> Also, there's already some code in fhandler_proc.cc, function
>>>> format_proc_cpuinfo to handle CPU groups.  You can use the
>>>> wincap.has_processor_groups() method to check if the system
>>>> supports CPU groups.
>>>
>>> Btw., Glibc's cpu_set_t supports up to 1024 CPUs.  See
>>> https://sourceware.org/git/?p=glibc.git;a=blob;f=posix/bits/cpu-set.h
>>> This may be ok for the foreseable future, I guess.
>>
>> Hi Corinna,
>> I will look into CPU group support; thanks for the pointers.  I also need to
>> fix the assumption I made about which flavor of pid would be handed to the
>> functions.. they will be Cygwin pids but need conversion to Windows pids
>> internally.
>
> Yeah, right, I missed to notice that.  I'll add a few notes inline
> over @ cygwin-patches.

I've updated my code locally to account for your notes on cygwin-patches; 
thanks!  I've also spent some time researching Windows affinities vs Linux 
affinities and have come to some conclusions.  I'm airing these for review
before I start coding in earnest.  I appreciate all comments from anybody 
interested.

(1) On Linux, one deals with processor affinities using a huge mask that 
allows to choose from all processors on the system.  On Windows, one deals 
with processor affinities for only the current processor group, max 64 
processors in a group.  This implies conversion between the two "views" 
when getting or setting processor affinities on Cygwin.

(2) On Linux, sched_get/setaffinity() take a pid_t argument, but it can 
be either a process id or a thread id.  If one selects a process id, the 
action affects just the main thread of that process.  On Windows, 
selecting the process id affects all threads of that process.

(3) For completeness, Linux's pthread_get/setaffinity_np() should probably 
be supplied by the proposed code too.

(4) I was looking at Cygwin's fhandler_proc.cc, function 
format_proc_cpuinfo().  There's a call to __get_cpus_per_group() which is 
implemented in miscfuncs.cc.  I haven't seen in the MSDN docs whether each 
processor group is guaranteed to have the same number of processors.  I 
might even expect variations on a NUMA system.  Anybody know if one can 
depend on the group membership of the first processor group to apply to 
all groups?

(5) On Windows, a process starts out in a particular processor group.  One 
can then change thread affinities in such a way that some threads run in a 
different processor group than other threads of the same process.  The 
process becomes a "multi-group" process.  This has implications for the 
conversions discussed in (1).

(6) On Linux, processor affinity is inherited across fork() and execve(). 
I'll need to ensure Cygwin's implementation of those calls handle affinity 
the same way.

So this is looking like a more substantial project :-).
Thanks for reading,

..mark