Failure during build of Python 3.8 via cygport

[Corinna Vinschen]

On Dec 14 02:42, Mark Geisert wrote:
> [Bringing this thread over from cygwin-apps...]
> 
> Mark Geisert wrote:
> > Mark Geisert wrote:
> > > This seems to be a problem setting up a platform-specific build
> > > directory. The sysconfig.py script wants to use "lib." + platform +
> > > pythonversion but the platform string somehow gets corrupted into
> > > non-utf8 bytes.  For instance, building Python 3.8 comes up with:
> > >      lib.cygwin-\365\377\377o-\377o-3.8
> > > as the directory name.  Broken, but could work.  The build failure
> > > happens because the script tries to write this directory name into a
> > > file but it's not a valid utf8 string.  The directory name should
> > > have been:
> > >      lib.cygwin-3.2.0-x86_64-3.8
> > 
> > And the corruption is due to something about a recent change to the
> > operation of Cygwin's uname() function.  The change was introduced in
> > Cygwin API version 335; I'm running 340 on my test machine.  This being
> > a fairly recent change might possibly explain why nobody else has run
> > into this issue yet.
> > 
> > Basically, os.uname within Python is calling Cygwin's uname() passing
> > the address of a buffer declared to be 'struct utsname'.  The structure
> > layout changed in API 335.  What I've hit is a mismatch between what
> > Python expects and Cygwin delivers.
> 
> Brian Inglis pointed out the API 335 change was made a couple years ago, so
> strike the quoted conjecture about why nobody else has hit this.  A new
> conjecture follows...
> 
> Debugging Python's os.uname showed Cygwin's uname() being called with a
> 'struct uname' as defined in /usr/include/sys/utsname.h, which is fine.  But
> it's the "old" pre-335 uname() interface being called, not the "new" 335+
> interface uname_x().  Note that the famous 'mkimport' script, used when
> building the Cygwin DLL, has an arg "--replace=uname=uname_x" which I
> believe is supposed to equate the two names so the code in uname_x() is
> called whether the interface is uname_x() or uname().  That's not happening.
> 
> My first thought was, dang it, my "optimizations" of mkimport broke it.  But
> that's not the case.  Restoring a previous version of mkimport doesn't help.
> 
> 'nm' shows that both uname and uname_x exist in libcygwin.a and also
> cygdll.a. And a newly-created Cygwin DLL has both functions, with different
> addresses.
> That's wrong, isn't it?

No, it isn't.  The old uname entry point is still required for old
applications built prior to API version 335.  If you build a new
application it will actually call uname_x when it tries to call uname
since that's how the libcygwin.a layer translates it.

It's a bit unfortunate that uname_x is exported by libcygwin.a
as well (in theory it should only export uname, pointing to uname_x
in the DLL), but this is how it always was, as you can see in
the 32 bit build with symbols only available there, e. g. aclcheck:

  $ nm libcygwin.a | grep aclcheck
  00000000 I __imp___aclcheck
  00000000 I __imp___aclcheck32
  00000000 I __imp__aclcheck
  00000000 T __aclcheck
	   U __imp___aclcheck
  00000000 T __aclcheck32
	   U __imp___aclcheck32
  00000000 T _aclcheck
	   U __imp___aclcheck32

In retrospect, uname_x should be named _uname_x or so, with a leading
underscore, so as not to pollute the namespace, but either way, that
isn;t your problem.

The problem here might be that you get the old uname function if
you dlopen the cygwin dll and dlsym(hdl, "uname").  Is that the
case in python?

If so, I have a simple, dirty workaround below.  Can you check if that's
the problem, please?


Corinna


>From 45519fc1faa616e18fc5e2d78c483699e0f1f475 Mon Sep 17 00:00:00 2001
From: Corinna Vinschen <corinna@vinschen.de>
Date: Mon, 14 Dec 2020 12:29:23 +0100
Subject: [PATCH] Cygwin: Make sure newer apps get uname_x even when loading
 uname dynamically

if an application built after API version 334 loads uname dynamically,
it actually gets the old uname, rather than the new uname_x.  Fix this by
checking the apps API version in uname and call uname_x instead, if it's
a newer app.

Signed-off-by: Corinna Vinschen <corinna@vinschen.de>
---
 winsup/cygwin/include/cygwin/version.h | 3 +++
 winsup/cygwin/uname.cc                 | 7 +++++++
 2 files changed, 10 insertions(+)

diff --git a/winsup/cygwin/include/cygwin/version.h b/winsup/cygwin/include/cygwin/version.h
index f21aba72669a..b39d0dbe95bb 100644
--- a/winsup/cygwin/include/cygwin/version.h
+++ b/winsup/cygwin/include/cygwin/version.h
@@ -74,6 +74,9 @@ details. */
 #define CYGWIN_VERSION_CHECK_FOR_EXTRA_TM_MEMBERS \
   (CYGWIN_VERSION_USER_API_VERSION_COMBINED >= 272)
 
+#define CYGWIN_VERSION_CHECK_FOR_UNAME_X \
+  (CYGWIN_VERSION_USER_API_VERSION_COMBINED >= 335)
+
 #define CYGWIN_VERSION_CYGWIN_CONV 181
 
 /* API_MAJOR 0.0: Initial version.  API_MINOR changes:
diff --git a/winsup/cygwin/uname.cc b/winsup/cygwin/uname.cc
index 350216681fa1..48f7dda6029b 100644
--- a/winsup/cygwin/uname.cc
+++ b/winsup/cygwin/uname.cc
@@ -92,7 +92,14 @@ struct old_utsname
 extern "C" int
 uname (struct utsname *in_name)
 {
+  /* This occurs if the application fetches the uname symbol dynamically.
+     We must call uname_x for newer API versions, otherwise the idea of
+     struct utsname doesn't match. */
+  if (CYGWIN_VERSION_CHECK_FOR_UNAME_X)
+    return uname_x (in_name);
+
   struct old_utsname *name = (struct old_utsname *) in_name;
+
   __try
     {
       char *snp = strstr  (cygwin_version.dll_build_date, "SNP");
-- 
2.29.2