Re: [netcdfgroup] Using NetCDF with OPeNDAP on MacOS Sierra (homebrew- and netCDF-dependant software)

  • To: Larry Baker <baker@xxxxxxxx>
  • Subject: Re: [netcdfgroup] Using NetCDF with OPeNDAP on MacOS Sierra (homebrew- and netCDF-dependant software)
  • From: "Capehart, William J" <William.Capehart@xxxxxxxxx>
  • Date: Thu, 4 May 2017 15:45:25 +0000
At Sean’s request I moved my .dodsrc I was able to get it to work.

(I think the only question I now have is what is wrong with that particular 
.dodsrc file..)

USE_CACHE=0
MAX_CACHE_SIZE=20
MAX_CACHED_OBJ=5
IGNORE_EXPIRES=0
CACHE_ROOT=/Users/wjc/.dods_cache/
DEFAULT_EXPIRES=86400
ALWAYS_VALIDATE=0
DEFLATE=0
VALIDATE_SSL=1
CURL.COOKIEJAR=.dods_cookies
CURL.SSL.VALIDATE=1
CURL.SSL.CERTIFICATE=/Users/wjc/.esg/credentials.pem
CURL.SSL.KEY=/Users/wjc/.esg/credentials.pem
CURL.SSL.CAPATH=/Users/wjc/.esg/certificates
HTTP.COOKIEJAR=/Users/wjc/.cookies
HTTP.NETRC=/Users/wjc/.netrc

Wjhich specific lines need to be edited or removed?

I’ll then close a second item that we have on github and close things there.

Thanks much all!


------------------------------------------------
Bill Capehart <William.Capehart@xxxxxxxxx<mailto:William.Capehart@xxxxxxxxx>>
Atmospheric and Environmental Sciences Program Coordinator
Civil and Environmental Engineering
201 Mineral Industries Building
South Dakota School of Mines and Technology
501 East St Joseph Street
Rapid City, SD 57701-3995
Ph: +1-605-394-1994   Mobile: +1-605-484-4692

From: Larry Baker <baker@xxxxxxxx>
Date: Thursday, May 4, 2017 at 00:10 MDT
To: William Capehart <William.Capehart@xxxxxxxxx>
Cc: "dmh@xxxxxxxx" <dmh@xxxxxxxx>, "netcdfgroup@xxxxxxxxxxxxxxxx" 
<netcdfgroup@xxxxxxxxxxxxxxxx>
Subject: Re: [netcdfgroup] Using NetCDF with OPeNDAP on MacOS Sierra (homebrew- 
and netCDF-dependant software)

See https://github.com/Unidata/netcdf-c/issues/122 around July 10, 2015 for a 
similar/the same bug?  My previous inference about strcmp() being the culprit 
is not quite right.  The failing code in the thread I found (below) is just 
before the strcmp() call.  Looks like the same problem to me.  The cause at 
that time was "c" had a value of 67, while the nameindices[] array was declared 
[26].

doutriaux1@maryam:[build]:[10587]> lldb  ~/build//install/Externals/bin/ncdump 
http://test.opendap.org/opendap/hyrax/netcdf/examples/ECMWF_ERA-40_subset.nc
(lldb) target create "/Users/doutriaux1/build//install/Externals/bin/ncdump"
Current executable set to 
'/Users/doutriaux1/build//install/Externals/bin/ncdump' (x86_64).
(lldb) settings set -- target.run-args  
"http://test.opendap.org/opendap/hyrax/netcdf/examples/ECMWF_ERA-40_subset.nc";
(lldb) r
Process 45302 launched: '/Users/doutriaux1/build//install/Externals/bin/ncdump' 
(x86_64)
Process 45302 stopped
* thread #1: tid = 0xebf1, 0x00000001000b09c7 
libnetcdf.7.dylib`occurlflagbyname(name=0x0000000100186749) + 327 at 
occurlflags.c:301, queue = 'com.apple.main-thread', stop reason = 
EXC_BAD_ACCESS (code=1, address=0x2)
    frame #0: 0x00000001000b09c7 
libnetcdf.7.dylib`occurlflagbyname(name=0x0000000100186749) + 327 at 
occurlflags.c:301
   298
   299      if(nameindices[c] == NULL)
   300      return NULL; /* no possible match */
-> 301      for(f=nameindices[c];f->name;f++) {
   302      int cmp = strcmp(name,f->name);
   303      if(cmp > 0) break; /* We assume sorted */
   304      if(cmp == 0) return f;

Larry Baker
US Geological Survey
650-329-5608
baker@xxxxxxxx<mailto:baker@xxxxxxxx>



On May 3, 2017, at 10:57 PM, Larry Baker 
<baker@xxxxxxxx<mailto:baker@xxxxxxxx>> wrote:

Looks to me like an illegal memory access in a strcmp() inside 
libnetcdf.11.dylib.  The "occurlflagbyname()" is the failing function inside 
libnetcdf?

Larry Baker
US Geological Survey
650-329-5608
baker@xxxxxxxx<mailto:baker@xxxxxxxx>



On May 3, 2017, at 10:44 PM, Capehart, William J 
<William.Capehart@xxxxxxxxx<mailto:William.Capehart@xxxxxxxxx>> wrote:

OK.  I am not familiar with how to use this but here is the output that I have 
for lldb usng …

lldb --debug  /usr/local/bin/ncdump  
http://test.opendap.org/opendap/data/nc/3fnoc.nc

https://gist.github.com/wjcapehart/8cc8068a6b6951ee98a2a48dbae02520

And for my output for the fortran program that I wrote.

https://gist.github.com/9a46b7750bc5a9c3ebfdf1916941f490


For the command

% otool -L /usr/local/bin/ncdump 
http://test.opendap.org/opendap/data/nc/3fnoc.nc
/usr/local/bin/ncdump:
                @rpath/libnetcdf.11.dylib (compatibility version 11.0.0, 
current version 11.4.0)
                /usr/local/opt/hdf5/lib/libhdf5_hl.100.dylib (compatibility 
version 101.0.0, current version 101.0.0)
                /usr/local/opt/hdf5/lib/libhdf5.100.dylib (compatibility 
version 101.0.0, current version 101.1.0)
                /usr/local/opt/szip/lib/libsz.2.dylib (compatibility version 
3.0.0, current version 3.0.0)
                /usr/local/lib/libz.1.dylib (compatibility version 1.0.0, 
current version 1.2.8)
                /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, 
current version 1238.50.2)
                /usr/lib/libcurl.4.dylib (compatibility version 7.0.0, current 
version 9.0.0)
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/objdump:
 'http://test.opendap.org/opendap/data/nc/3fnoc.nc': No such file or directory


------------------------------------------------
Bill Capehart <William.Capehart@xxxxxxxxx<mailto:William.Capehart@xxxxxxxxx>>
Atmospheric and Environmental Sciences Program Coordinator
Civil and Environmental Engineering
201 Mineral Industries Building
South Dakota School of Mines and Technology
501 East St Joseph Street
Rapid City, SD 57701-3995
Ph: +1-605-394-1994   Mobile: +1-605-484-4692

From: 
<netcdfgroup-bounces@xxxxxxxxxxxxxxxx<mailto:netcdfgroup-bounces@xxxxxxxxxxxxxxxx>>
 on behalf of Larry Baker <baker@xxxxxxxx<mailto:baker@xxxxxxxx>>
Date: Wednesday, May 3, 2017 at 16:45 MDT
To: "dmh@xxxxxxxx<mailto:dmh@xxxxxxxx>" <dmh@xxxxxxxx<mailto:dmh@xxxxxxxx>>
Cc: "netcdfgroup@xxxxxxxxxxxxxxxx<mailto:netcdfgroup@xxxxxxxxxxxxxxxx>" 
<netcdfgroup@xxxxxxxxxxxxxxxx<mailto:netcdfgroup@xxxxxxxxxxxxxxxx>>
Subject: Re: [netcdfgroup] Using NetCDF with OPeNDAP on MacOS Sierra (homebrew- 
and netCDF-dependant software)

Good idea: lldb

Larry Baker
US Geological Survey
650-329-5608
baker@xxxxxxxx<mailto:baker@xxxxxxxx>





On 3 May 2017, at 3:35:19 PM, dmh@xxxxxxxx<mailto:dmh@xxxxxxxx> wrote:

Is there a osx equivalent of gdb? If so, you could
probably run the ncdump command using that debugger
and it will probably catch the seg fault. This would
allow some idea of where the failure is occurring.
=Dennis Heimbigner
Unidata

On 5/3/2017 4:25 PM, Capehart, William J wrote:


To be desperate ;-P is there a way to test things with just the libcurl to see 
where the problem is?   I am assuming that most people are getting satisfaction 
with the resident version with Sierra.  I'm really on my last leg here.  I'm 
not even getting core files with my seg faults on ncdump or my simple Fortran 
program inbuilt to test it...  both run just fine on Centos .
Sent from my iPhone


On May 3, 2017, at 15:04, "dmh@xxxxxxxx<mailto:dmh@xxxxxxxx>" 
<dmh@xxxxxxxx<mailto:dmh@xxxxxxxx>> wrote:

To be pedantic, ncdump (via the netcdf-c library) uses libcurl.
=Dennis Heimbigner
Unidata



On 5/3/2017 2:31 PM, Roy Mendelssohn - NOAA Federal wrote:
Hmm - I may be wrong but i thought the ncdump for a remote file just uses curl 
to get  the .das and .dds and forms the .cdl response from those.  Have you 
tried a different server?
Say,
ncdump -h http://coastwatch.pfeg.noaa.gov/erddap/griddap/jplAquariusSSSMonthlyV4
-Roy


On May 3, 2017, at 1:25 PM, Capehart, William J 
<William.Capehart@xxxxxxxxx<mailto:William.Capehart@xxxxxxxxx>> wrote:

Roy for the curl I get the followingf

% curl -v http://test.opendap.org/opendap/data/nc/3fnoc.nc.das
*   Trying 52.44.0.41...
* TCP_NODELAY set
* Connected to test.opendap.org<http://test.opendap.org/> (52.44.0.41) port 80 
(#0)


GET /opendap/data/nc/3fnoc.nc.das HTTP/1.1
Host: test.opendap.org<http://test.opendap.org/>
User-Agent: curl/7.51.0
Accept: */*
< HTTP/1.1 200 OK
< Date: Wed, 03 May 2017 20:22:21 GMT
< X-FRAME-OPTIONS: DENY
< Last-Modified: Tue, 10 Jan 2006 18:32:26 GMT
< Set-Cookie: JSESSIONID=63E4296077ED5DBE9CFE2C49B169D4ED; Path=/opendap; 
HttpOnly
< XDODS-Server: dods/3.2
< XOPeNDAP-Server: bes/3.17.4, csv_handler/1.1.6, dap-server/ascii/4.1.5, 
dap-server/usage/4.2.6, dap-server/www/4.1.5, dapreader_module/0.0.1, 
fileout_gdal/0.10.1, fileout_json/1.0.6, fileout_netcdf/1.4.3, 
fits_handler/1.0.18, freeform_handler/3.9.6, functions/1.1.0, 
gateway_module/1.1.9, gdal_handler/1.0.7, hdf4_handler/3.12.3, 
hdf5_handler/2.3.5, libdap/3.18.3, ncml_moddule/1.4.4, netcdf_handler/3.11.6, 
w10n_handler/1.0.6, xml_data_handler/1.1.2
< X-DAP: 3.2
< Content-Description: dods_das
< Content-Type: text/plain
< Connection: close
< Transfer-Encoding: chunked
<
Attributes {
   u {
       String units "meter per second";
       String long_name "Vector wind eastward component";
       String missing_value "-32767";
       String scale_factor "0.005";
   }
   v {
       String units "meter per second";
       String long_name "Vector wind northward component";
       String missing_value "-32767";
       String scale_factor "0.005";
   }
   lat {
       String units "degree North";
   }
   lon {
       String units "degree East";
   }
   time {
       String units "hours from base_time";
   }
   NC_GLOBAL {
       String base_time "88-245-00:00:00";
       String title " FNOC UV wind components from 1988-245 to 1988-247.";
   }
   DODS_EXTRA {
       String Unlimited_Dimension "time_a";
   }
}
* Curl_http_done: called premature == 0
* Closing connection 0


------------------------------------------------
Bill Capehart <William.Capehart@xxxxxxxxx<mailto:William.Capehart@xxxxxxxxx>>
Atmospheric and Environmental Sciences Program Coordinator
Civil and Environmental Engineering
201 Mineral Industries Building
South Dakota School of Mines and Technology
501 East St Joseph Street
Rapid City, SD 57701-3995
Ph: +1-605-394-1994   Mobile: +1-605-484-4692

On 5/3/17, 14:19 MDT, "Roy Mendelssohn - NOAA Federal" 
<roy.mendelssohn@xxxxxxxx<mailto:roy.mendelssohn@xxxxxxxx>> wrote:

   More specifically,  what do you get when you give:

   curl -v http://test.opendap.org/opendap/data/nc/3fnoc.nc.das


   -Roy




On May 3, 2017, at 1:01 PM, Roy Mendelssohn - NOAA Federal 
<roy.mendelssohn@xxxxxxxx<mailto:roy.mendelssohn@xxxxxxxx>> wrote:

What happens if you just do a curl to get the address but ask for the .das 
instead,  and see if your curl and access the site.

-Roy



On May 3, 2017, at 12:57 PM, Capehart, William J 
<William.Capehart@xxxxxxxxx<mailto:William.Capehart@xxxxxxxxx>> wrote:

Did that with both fink and homebrew including compiling all from source each 
time.  Still no traction.  :-(

------------------------------------------------
Bill Capehart <William.Capehart@xxxxxxxxx<mailto:William.Capehart@xxxxxxxxx>>
Atmospheric and Environmental Sciences Program Coordinator
Civil and Environmental Engineering
201 Mineral Industries Building
South Dakota School of Mines and Technology
501 East St Joseph Street
Rapid City, SD 57701-3995
Ph: +1-605-394-1994   Mobile: +1-605-484-4692

On 5/3/17, 08:06 MDT, "Roy Mendelssohn - NOAA Federal" 
<roy.mendelssohn@xxxxxxxx<mailto:roy.mendelssohn@xxxxxxxx>> wrote:

 try installing the fink curl.

 -roy



On May 3, 2017, at 6:47 AM, Capehart, William J 
<William.Capehart@xxxxxxxxx<mailto:William.Capehart@xxxxxxxxx>> wrote:

I am attaching my gist of the build

https://gist.github.com/anonymous/4db828fcd4384c7883a8687147ced1f7

The C tst is 3.make
The C++4 Test is 7.make
The Fortran Test is 11.make
The C++ test is 16.make

Hi Sean: And the output of my nc-config –alll

https://gist.github.com/733ad6330523a83cee204f9f7f34c061


The curl version on my mac (10.12.4) is the on-board version which I presume is 
what I need

%  curl --version
curl 7.51.0 (x86_64-apple-darwin16.0) libcurl/7.51.0 SecureTransport zlib/1.2.8
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 
pop3s rtsp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL 
libz UnixSockets

And curl -Oing my the target file produces the following

https://gist.github.com/wjcapehart/6f49640a075f47b1040c48917ebd631b

Sorry Larry:  valgrind is reporting from homebrew that is in compatible with 
MacOS post El Capitan.

Roy: from the client side I’m not seeing any significant redflags in the 
console logs

Really really out of clues at this point.  Thanks to everyone or their input 
though.

Bill


From: Sean Arms <sarms@xxxxxxxx<mailto:sarms@xxxxxxxx>>
Date: Tuesday, May 2, 2017 at 17:50 MDT
To: William Capehart 
<William.Capehart@xxxxxxxxx<mailto:William.Capehart@xxxxxxxxx>>
Cc: "netcdfgroup@xxxxxxxxxxxxxxxx<mailto:netcdfgroup@xxxxxxxxxxxxxxxx>" 
<netcdfgroup@xxxxxxxxxxxxxxxx<mailto:netcdfgroup@xxxxxxxxxxxxxxxx>>
Subject: Re: [netcdfgroup] Using NetCDF with OPeNDAP on MacOS Sierra (homebrew- 
and netCDF-dependant software)

Greetings Bill,

I have a suspicion this has to do with the way curl was built. What does

nc-config --libs

give you?

Sean




On Sun, Apr 30, 2017 at 10:42 AM, Capehart, William J 
<William.Capehart@xxxxxxxxx<mailto:William.Capehart@xxxxxxxxx>> wrote:
Hi All

I have been having a particularly time working with NetCDF with OPeNDAP on my 
Macs (right now both are on MacOS 10.12.4).  I’ve mostly been using homebrew 
but this also happens with NCL 6.3 and 6.4 as downloaded from Earth System Grid.

When testing code using just the NetCDF fortran libraries as well as just using 
ncdump I get segmentation faults when I try to access 
http://test.opendap.org/opendap/data/nc/3fnoc.nc

Is anyone else having any luck with their MacOS Sierra NetCDF builds and the 
software that leverages NetCDF?


------------------------------------------------
Bill Capehart <William.Capehart@xxxxxxxxx<mailto:William.Capehart@xxxxxxxxx>>
Atmospheric and Environmental Sciences Program Coordinator
Civil and Environmental Engineering
201 Mineral Industries Building
South Dakota School of Mines and Technology
501 East St Joseph Street
Rapid City, SD 57701-3995
Ph: +1-605-394-1994   Mobile: +1-605-484-4692


_______________________________________________
NOTE: All exchanges posted to Unidata maintained email lists are
recorded in the Unidata inquiry tracking system and made publicly
available through the web.  Users who post to any of the lists we
maintain are reminded to remove any personal information that they
do not want to be made public.


netcdfgroup mailing list
netcdfgroup@xxxxxxxxxxxxxxxx<mailto:netcdfgroup@xxxxxxxxxxxxxxxx>
For list information or to unsubscribe,  visit: 
http://www.unidata.ucar.edu/mailing_lists/


 **********************
 "The contents of this message do not reflect any position of the U.S. 
Government or NOAA."
 **********************
 Roy Mendelssohn
 Supervisory Operations Research Analyst
 NOAA/NMFS
 Environmental Research Division
 Southwest Fisheries Science Center
 ***Note new street address***
 110 McAllister Way
 Santa Cruz, CA 95060
 Phone: (831)-420-3666
 Fax: (831) 420-3980
 e-mail: Roy.Mendelssohn@xxxxxxxx<mailto:Roy.Mendelssohn@xxxxxxxx> www: 
http://www.pfeg.noaa.gov/

 "Old age and treachery will overcome youth and skill."
 "From those who have been given much, much will be expected"
 "the arc of the moral universe is long, but it bends toward justice" -MLK Jr.




**********************
"The contents of this message do not reflect any position of the U.S. 
Government or NOAA."
**********************
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
***Note new street address***
110 McAllister Way
Santa Cruz, CA 95060
Phone: (831)-420-3666
Fax: (831) 420-3980
e-mail: Roy.Mendelssohn@xxxxxxxx<mailto:Roy.Mendelssohn@xxxxxxxx> www: 
http://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected"
"the arc of the moral universe is long, but it bends toward justice" -MLK Jr.

   **********************
   "The contents of this message do not reflect any position of the U.S. 
Government or NOAA."
   **********************
   Roy Mendelssohn
   Supervisory Operations Research Analyst
   NOAA/NMFS
   Environmental Research Division
   Southwest Fisheries Science Center
   ***Note new street address***
   110 McAllister Way
   Santa Cruz, CA 95060
   Phone: (831)-420-3666
   Fax: (831) 420-3980
   e-mail: Roy.Mendelssohn@xxxxxxxx<mailto:Roy.Mendelssohn@xxxxxxxx> www: 
http://www.pfeg.noaa.gov/

   "Old age and treachery will overcome youth and skill."
   "From those who have been given much, much will be expected"
   "the arc of the moral universe is long, but it bends toward justice" -MLK Jr.



**********************
"The contents of this message do not reflect any position of the U.S. 
Government or NOAA."
**********************
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
***Note new street address***
110 McAllister Way
Santa Cruz, CA 95060
Phone: (831)-420-3666
Fax: (831) 420-3980
e-mail: Roy.Mendelssohn@xxxxxxxx<mailto:Roy.Mendelssohn@xxxxxxxx> www: 
http://www.pfeg.noaa.gov/
"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected"
"the arc of the moral universe is long, but it bends toward justice" -MLK Jr.
_______________________________________________
NOTE: All exchanges posted to Unidata maintained email lists are
recorded in the Unidata inquiry tracking system and made publicly
available through the web.  Users who post to any of the lists we
maintain are reminded to remove any personal information that they
do not want to be made public.
netcdfgroup mailing list
netcdfgroup@xxxxxxxxxxxxxxxx<mailto:netcdfgroup@xxxxxxxxxxxxxxxx>
For list information or to unsubscribe,  visit: 
http://www.unidata.ucar.edu/mailing_lists/

_______________________________________________
NOTE: All exchanges posted to Unidata maintained email lists are
recorded in the Unidata inquiry tracking system and made publicly
available through the web.  Users who post to any of the lists we
maintain are reminded to remove any personal information that they
do not want to be made public.


netcdfgroup mailing list
netcdfgroup@xxxxxxxxxxxxxxxx<mailto:netcdfgroup@xxxxxxxxxxxxxxxx>
For list information or to unsubscribe,  visit: 
http://www.unidata.ucar.edu/mailing_lists/

_______________________________________________
NOTE: All exchanges posted to Unidata maintained email lists are
recorded in the Unidata inquiry tracking system and made publicly
available through the web.  Users who post to any of the lists we
maintain are reminded to remove any personal information that they
do not want to be made public.


netcdfgroup mailing list
netcdfgroup@xxxxxxxxxxxxxxxx<mailto:netcdfgroup@xxxxxxxxxxxxxxxx>
For list information or to unsubscribe,  visit: 
http://www.unidata.ucar.edu/mailing_lists/



  • 2017 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdfgroup archives: