From 39f481ea6f4d22f50afa67250331c7ec90081384 Mon Sep 17 00:00:00 2001 From: Fletcher Dunn Date: Wed, 17 Nov 2021 17:25:21 -0800 Subject: [PATCH] Improved vcpkg support Moved instructions for building windows manually to a separate file and marked that method as not supported. - Deleted the vcpkg CONTROL file, upgrade to a modern vcpkg.json manifest. - Added tests and examples as vcpkg "features". - Make sure we obey static/shared from the vcpkg triplet (I believe this addresses #175) - Split out the detailed instructions for how to build Windows dependencies manually to a separate document, so we don't clutter up the main doc. Also: - Examples will always link with shared lib - Tests always link with the static lib I'm doing my best here to obey vcpkg (and cmake in general) best practices / conventions, but this is not really something I have expertise in, and the documentation on all this is realy not good, so feel free to make suggestions if you see something that could be done better. Possible future improvements (HELP WANTED): - Don't select the crypto with a feature, that is explicitly listed in the vcpkg docs as an anti-pattern. - Add a feature for the cert tool. - Add more CI actions to build more platforms to make sure they are all working. --- BUILDING.md | 165 +++++++++--------- BUILDING_WINDOWS_MANUAL.md | 119 +++++++++++++ CMakeLists.txt | 10 +- examples/CMakeLists.txt | 2 +- tests/CMakeLists.txt | 10 +- vcpkg_ports/gamenetworkingsockets/CONTROL | 18 -- .../gamenetworkingsockets/portfile.cmake | 56 ++++-- vcpkg_ports/gamenetworkingsockets/vcpkg.json | 29 +++ 8 files changed, 291 insertions(+), 118 deletions(-) create mode 100644 BUILDING_WINDOWS_MANUAL.md delete mode 100644 vcpkg_ports/gamenetworkingsockets/CONTROL create mode 100644 vcpkg_ports/gamenetworkingsockets/vcpkg.json diff --git a/BUILDING.md b/BUILDING.md index 97085d4..ed79a32 100644 --- a/BUILDING.md +++ b/BUILDING.md @@ -15,7 +15,8 @@ Building minor changes, so the source is included in this project.) * libsodium * [bcrypt](https://docs.microsoft.com/en-us/windows/desktop/api/bcrypt/) - (windows only) + (Windows only. Note the primary reason this is supported is to satisfy + an Xbox requuirement.) * Google protobuf 2.6.1+ * Google [webrtc](https://opensource.google/projects/webrtc) is used for NAT piercing (ICE) for P2P connections. The relevant code is linked in as a @@ -64,6 +65,8 @@ The following instructions assume that you will follow the vcpkg recommendations vcpkg as a subfolder. If you want to install vcpkg somewhere else, you're on your own. See the [quick start](https://github.com/microsoft/vcpkg/#quick-start-windows) for more info. +If you don't want to use vcpkg, try the [manual instructions](BUILDING_WINDOWS_MANUAL.md) + First, bootstrap vcpkg. From the root folder of your GameNetworkingSockets workspace: ``` @@ -84,9 +87,89 @@ The library should be immediately available in Visual Studio projects if the vcpkg integration is installed, or the vcpkg CMake toolchain file can be used for CMake-based projects. -### Manual setup +## Mac OS X -Setting up the dependencies by hand is a bit of an arduous gauntlet. +Using [Homebrew](https://brew.sh) + +### OpenSSL + +``` +$ brew install openssl +$ export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/usr/local/opt/openssl/lib/pkgconfig +``` +GameNetworkingSockets requries openssl version 1.1+, so if you install and link openssl but at compile you see the error ```Dependency libcrypto found: NO (tried cmake and framework)``` you'll need to force Brew to install openssl 1.1. You can do that like this: +``` +$ brew install openssl@1.1 +$ export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/usr/local/opt/openssl@1.1/lib/pkgconfig +``` + +### protobuf + +``` +$ brew install protobuf +``` + +## MSYS2 + +You can also build this project on [MSYS2](https://www.msys2.org). First, +follow the [instructions](https://github.com/msys2/msys2/wiki/MSYS2-installation) on the +MSYS2 website for updating your MSYS2 install. + +**Be sure to follow the instructions at the site above to update MSYS2 before +you continue. A fresh install is *not* up to date by default.** + +Next install the dependencies for building GameNetworkingSockets (if you want +a 32-bit build, install the i686 versions of these packages): + +``` +$ pacman -S \ + git \ + mingw-w64-x86_64-gcc \ + mingw-w64-x86_64-openssl \ + mingw-w64-x86_64-pkg-config \ + mingw-w64-x86_64-protobuf +``` + +And finally, clone the repository and build it: + +``` +$ git clone https://github.com/ValveSoftware/GameNetworkingSockets.git +$ cd GameNetworkingSockets +$ mkdir build +$ cd build +$ cmake -G Ninja .. +$ ninja +``` + +**NOTE:** When building with MSYS2, be sure you launch the correct version of +the MSYS2 terminal, as the three different Start menu entries will give you +different environment variables that will affect the build. You should run the +Start menu item named `MSYS2 MinGW 64-bit` or `MSYS2 MinGW 32-bit`, depending +on the packages you've installed and what architecture you want to build +GameNetworkingSockets for. + + +## Visual Studio Code +If you're using Visual Studio Code, we have a few extensions to recommend +installing, which will help build the project. Once you have these extensions +installed, open up the .code-workspace file in Visual Studio Code. + +### C/C++ by Microsoft +This extension provides IntelliSense support for C/C++. + +VS Marketplace Link: https://marketplace.visualstudio.com/items?itemName=ms-vscode.cpptools + +### CMake Tools by vector-of-bool +This extension allows for configuring the CMake project and building it from +within the Visual Studio Code IDE. + +VS Marketplace Link: https://marketplace.visualstudio.com/items?itemName=vector-of-bool.cmake-tools + +### Windows Manual setup + +If you are on Windows and don't want to use vcpkg, you might try these instructions. +This is a bit of an arduous gauntlet, and this method is no longer supported, but +we didn't delete these instructions in case somebody finds them useful. #### OpenSSL @@ -197,80 +280,4 @@ Finally, perform the build C:\dev\GameNetworkingSockets\build> ninja ``` -## Mac OS X -Using [Homebrew](https://brew.sh) - -### OpenSSL - -``` -$ brew install openssl -$ export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/usr/local/opt/openssl/lib/pkgconfig -``` -GameNetworkingSockets requries openssl version 1.1+, so if you install and link openssl but at compile you see the error ```Dependency libcrypto found: NO (tried cmake and framework)``` you'll need to force Brew to install openssl 1.1. You can do that like this: -``` -$ brew install openssl@1.1 -$ export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/usr/local/opt/openssl@1.1/lib/pkgconfig -``` - -### protobuf - -``` -$ brew install protobuf -``` - -## MSYS2 - -You can also build this project on [MSYS2](https://www.msys2.org). First, -follow the [instructions](https://github.com/msys2/msys2/wiki/MSYS2-installation) on the -MSYS2 website for updating your MSYS2 install. - -**Be sure to follow the instructions at the site above to update MSYS2 before -you continue. A fresh install is *not* up to date by default.** - -Next install the dependencies for building GameNetworkingSockets (if you want -a 32-bit build, install the i686 versions of these packages): - -``` -$ pacman -S \ - git \ - mingw-w64-x86_64-gcc \ - mingw-w64-x86_64-openssl \ - mingw-w64-x86_64-pkg-config \ - mingw-w64-x86_64-protobuf -``` - -And finally, clone the repository and build it: - -``` -$ git clone https://github.com/ValveSoftware/GameNetworkingSockets.git -$ cd GameNetworkingSockets -$ mkdir build -$ cd build -$ cmake -G Ninja .. -$ ninja -``` - -**NOTE:** When building with MSYS2, be sure you launch the correct version of -the MSYS2 terminal, as the three different Start menu entries will give you -different environment variables that will affect the build. You should run the -Start menu item named `MSYS2 MinGW 64-bit` or `MSYS2 MinGW 32-bit`, depending -on the packages you've installed and what architecture you want to build -GameNetworkingSockets for. - - -## Visual Studio Code -If you're using Visual Studio Code, we have a few extensions to recommend -installing, which will help build the project. Once you have these extensions -installed, open up the .code-workspace file in Visual Studio Code. - -### C/C++ by Microsoft -This extension provides IntelliSense support for C/C++. - -VS Marketplace Link: https://marketplace.visualstudio.com/items?itemName=ms-vscode.cpptools - -### CMake Tools by vector-of-bool -This extension allows for configuring the CMake project and building it from -within the Visual Studio Code IDE. - -VS Marketplace Link: https://marketplace.visualstudio.com/items?itemName=vector-of-bool.cmake-tools diff --git a/BUILDING_WINDOWS_MANUAL.md b/BUILDING_WINDOWS_MANUAL.md new file mode 100644 index 0000000..1093062 --- /dev/null +++ b/BUILDING_WINDOWS_MANUAL.md @@ -0,0 +1,119 @@ +Building Windows w/out Using vcpkg +--- + +We recommend using vcpkg to get the dependencies installed. But if that +doesn't work for you for some reason, you might try these instructions. +This is a bit of an arduous gauntlet, and this method is no longer supported, +but since these instructions were written, and at the present time still work, +we didn't delete them in case somebody finds them useful. However, this is +not a supported method of building the library so please don't file any issues. + +#### Checking prerequisites + +Start a Visual Studio Command Prompt (2017+), and double-check +that you have everything you need. Note that Visual Studio comes with these tools, +but you might not have selected to install them. Or just install them from somewhere +else and put them in your `PATH`. + +*IMPORTANT*: Make sure you start the command prompt for the desired target +architecture (x64 or x64)! In the examples here we are building 64-bit. + +``` +********************************************************************** +** Visual Studio 2019 Developer Command Prompt v16.5.4 +** Copyright (c) 2019 Microsoft Corporation +********************************************************************** +[vcvarsall.bat] Environment initialized for: 'x64' + +C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise> cd \dev + +C:\dev> git --version +git version 2.17.1.windows.2 + +C:\dev> cmake --version # 3.5 or higher is required +cmake version 3.16.19112601-MSVC_2 + +C:\dev> ninja --version +1.8.2 +``` + +#### OpenSSL + +You can install the [OpenSSL binaries](https://slproweb.com/products/Win32OpenSSL.html) +provided by Shining Light Productions. The Windows CMake distribution understands +how to find the OpenSSL binaries from these installers, which makes building a lot +easier. Be sure to pick the installers **without** the "Light"suffix. In this instance, +"Light" means no development libraries or headers. + +For CMake to find the libraries, you may need to set the environment variable +`OPENSSL_ROOT_DIR`. + +#### Protobuf + +Instructions for getting a working installation of google protobuf on Windows can +be found [here](https://github.com/protocolbuffers/protobuf/blob/master/cmake/README.md). + +Here is an example. First, start a Visual Studio Command Prompt as above. Then download +a particular release of the source. Here we are using `git`, but +you can also just download a [release .zip](https://github.com/protocolbuffers/protobuf/releases). + +``` +C:\dev> git clone -b 3.5.x https://github.com/google/protobuf +C:\dev> cd protobuf +``` + +Compile the protobuf source. You need to make sure that all of the following match +the settings you will use for compiling GameNetworkingSockets: + +* The target architecture must match (controlled by the MSVC environment variables). +* ```CMAKE_BUILD_TYPE```, which controls debug or release for both projects, + and must match. +* ```protobuf_BUILD_SHARED_LIBS=ON``` in the example indicates that + GameNetworkingSockets will link dynamically with protobuf .dlls, which is the + default for GameNetworkingSockets. For static linkage, remove this and set + ``Protobuf_USE_STATIC_LIBS=ON`` when building GameNetworkingSockets. +* If you link statically with protobuf, then you will also need to make sure that + the linkage with the MSVC CRT is the same. The default for both protobuf and + GameNetworkingSockets is multithreaded dll. + +Also, note the value for ```CMAKE_INSTALL_PREFIX```. This specifies where to +"install" the library (headers, link libraries, and the protoc compiler tool). + +``` +C:\dev\protobuf> mkdir cmake_build +C:\dev\protobuf> cd cmake_build +C:\dev\protobuf\cmake_build> cmake -G Ninja -DCMAKE_BUILD_TYPE=Release -Dprotobuf_BUILD_TESTS=OFF -Dprotobuf_BUILD_SHARED_LIBS=ON -DCMAKE_INSTALL_PREFIX=c:\sdk\protobuf-amd64 ..\cmake +C:\dev\protobuf\cmake_build> ninja +C:\dev\protobuf\cmake_build> ninja install +``` + +#### Building + +Start a Visual Studio Command Prompt, and create a directory to hold the build output. + +``` +C:\dev\GameNetworkingSockets> mkdir build +C:\dev\GameNetworkingSockets> cd build +``` + +You'll need to add the path to the protobuf `bin` folder to your path, so +CMake can find the protobuf compiler. If you followed the example above, that would +be something like this: + +``` +C:\dev\GameNetworkingSockets\build> set PATH=%PATH%;C:\sdk\protobuf-amd64\bin +``` + +Now invoke cmake to generate the type or project you want to build. Here we are creating +ninja files, for a 100% command line build. It's also possible to get cmake to output +Visual studio project (`.vcxproj`) and solution (`.sln`) files. +``` +C:\dev\GameNetworkingSockets\build> cmake -G Ninja -DCMAKE_BUILD_TYPE=Release .. +``` + +Finally, perform the build +``` +C:\dev\GameNetworkingSockets\build> ninja +``` + + diff --git a/CMakeLists.txt b/CMakeLists.txt index 615c35d..bbcf98f 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -62,7 +62,7 @@ set(useCryptoOptions OpenSSL libsodium BCrypt) set(USE_CRYPTO "OpenSSL" CACHE STRING "Crypto library to use for AES/SHA256") set_property(CACHE USE_CRYPTO PROPERTY STRINGS ${useCryptoOptions}) -list(FIND useCryptoOptions ${USE_CRYPTO} useCryptoIndex) +list(FIND useCryptoOptions "${USE_CRYPTO}" useCryptoIndex) if(useCryptoIndex EQUAL -1) message(FATAL_ERROR "USE_CRYPTO must be one of: ${useCryptoOptions}") endif() @@ -217,10 +217,18 @@ function(set_target_common_gns_properties TGT) endfunction() if(BUILD_EXAMPLES) + if ( NOT BUILD_SHARED_LIB ) + # See also portfile.cmake + message(FATAL_ERROR "Must build shared lib (-DBUILD_SHARED_LIB=ON) to build examples") + endif() add_subdirectory(examples) endif() if(BUILD_TESTS) + if ( NOT BUILD_STATIC_LIB ) + # See also portfile.cmake + message(FATAL_ERROR "Must build static lib (-DBUILD_STATIC_LIB=ON) to build tests") + endif() add_subdirectory(tests) endif() diff --git a/examples/CMakeLists.txt b/examples/CMakeLists.txt index 30e6d41..0d15967 100644 --- a/examples/CMakeLists.txt +++ b/examples/CMakeLists.txt @@ -43,7 +43,7 @@ if(${CMAKE_PROJECT_NAME} STREQUAL "gns_example") find_package(GameNetworkingSockets CONFIG REQUIRED) endif() -target_link_libraries(example_chat GameNetworkingSockets::GameNetworkingSockets) +target_link_libraries(example_chat GameNetworkingSockets::shared) if(COMMAND add_sanitizers) add_sanitizers(example_chat) diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index 4c0c4d3..4cabaf3 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -1,10 +1,12 @@ if(SANITIZE) add_definitions(-DSANITIZER) - set( GAMENETWORKINGSOCKETS_LIB GameNetworkingSockets_s ) -else() - set( GAMENETWORKINGSOCKETS_LIB GameNetworkingSockets ) endif() +# Let's always link all tests statically with the lib +# To keep things simple. (Since some of the tests MUST +# link statically) +set( GAMENETWORKINGSOCKETS_LIB "GameNetworkingSockets::static" ) + add_executable( test_connection test_common.cpp @@ -19,7 +21,7 @@ add_executable( ) set_target_common_gns_properties( test_crypto ) target_include_directories(test_crypto PRIVATE ../src ../src/public ../src/common ../include) -target_link_libraries(test_crypto GameNetworkingSockets_s) +target_link_libraries(test_crypto GameNetworkingSockets::static) add_sanitizers(test_crypto) # Test data for the crypto test when the project is built diff --git a/vcpkg_ports/gamenetworkingsockets/CONTROL b/vcpkg_ports/gamenetworkingsockets/CONTROL deleted file mode 100644 index 4383350..0000000 --- a/vcpkg_ports/gamenetworkingsockets/CONTROL +++ /dev/null @@ -1,18 +0,0 @@ -Source: gamenetworkingsockets -Version: 2020-10-06 -Description: GameNetworkingSockets is a basic transport layer for games. -Homepage: https://github.com/ValveSoftware/GameNetworkingSockets -Build-Depends: protobuf -Default-Features: openssl - -Feature: openssl -Description: Use OpenSSL as the crypto backend -Build-Depends: openssl - -Feature: libsodium -Description: Use libsodium as the crypto backend -Build-Depends: libsodium - -Feature: webrtc -Description: Compiles WebRTC support for P2P -Build-Depends: abseil diff --git a/vcpkg_ports/gamenetworkingsockets/portfile.cmake b/vcpkg_ports/gamenetworkingsockets/portfile.cmake index ef279a6..e7c18e9 100644 --- a/vcpkg_ports/gamenetworkingsockets/portfile.cmake +++ b/vcpkg_ports/gamenetworkingsockets/portfile.cmake @@ -1,37 +1,63 @@ set(SOURCE_PATH "${CMAKE_CURRENT_LIST_DIR}/../..") -if(openssl IN_LIST FEATURES) - set(CRYPTO_BACKEND OpenSSL) -endif() - -if(libsodium IN_LIST FEATURES) - set(CRYPTO_BACKEND libsodium) +# Select crypto backend based on the selected crypto "feature" +# WE ARE NOT SUPPOSED TO BE DOING THIS. +# I'd be super happy if somebody wants to fix this up properly. +if ("libsodium" IN_LIST FEATURES) + set(CRYPTO_BACKEND "libsodium") +endif() +if ("bcrypt" IN_LIST FEATURES) + set(CRYPTO_BACKEND "BCrypt") +endif() +if ( ( "${CRYPTO_BACKEND}" STREQUAL "" ) OR ( "openssl" IN_LIST FEATURES ) ) + set(CRYPTO_BACKEND "OpenSSL") endif() +# vcpkg_check_features( OUT_FEATURE_OPTIONS FEATURE_OPTIONS FEATURES - webrtc USE_STEAMWEBRTC + webrtc USE_STEAMWEBRTC + examples BUILD_EXAMPLES + tests BUILD_TESTS ) +# Check static versus dynamic in the triple. Our cmakefile can build both +# of them, but in the context of vcpkg, we will only build one or the other +string(COMPARE EQUAL "${VCPKG_LIBRARY_LINKAGE}" "dynamic" BUILD_SHARED_LIB) +string(COMPARE EQUAL "${VCPKG_LIBRARY_LINKAGE}" "static" BUILD_STATIC_LIB) + +# Select how to link the MSVC C runtime lib. When building the static +# lib, we will link the CRT statically. Otherwise, link dynamically. +string(COMPARE EQUAL "${VCPKG_LIBRARY_LINKAGE}" "static" MSVC_CRT_STATIC) + +if ( BUILD_EXAMPLES AND NOT BUILD_SHARED_LIB ) + message(FATAL_ERROR "Examples must be built with shared linkage") +endif() +if ( BUILD_TESTS AND NOT BUILD_STATIC_LIB ) + message(FATAL_ERROR "Tests must be built with static linkage") +endif() + vcpkg_configure_cmake( SOURCE_PATH ${SOURCE_PATH} PREFER_NINJA OPTIONS - -DGAMENETWORKINGSOCKETS_BUILD_TESTS=OFF - -DGAMENETWORKINGSOCKETS_BUILD_EXAMPLES=OFF -DUSE_CRYPTO=${CRYPTO_BACKEND} - -DUSE_CRYPTO25519=${CRYPTO_BACKEND} + -DBUILD_STATIC_LIB=${BUILD_STATIC_LIB} + -DBUILD_SHARED_LIB=${BUILD_SHARED_LIB} + -DMSVC_CRT_STATIC=${MSVC_CRT_STATIC} ${FEATURE_OPTIONS} ) vcpkg_install_cmake() +#vcpkg_fixup_cmake_targets(CONFIG_PATH "lib/cmake/GameNetworkingSockets") + +# Copy some files + file(INSTALL ${SOURCE_PATH}/LICENSE DESTINATION ${CURRENT_PACKAGES_DIR}/share/${PORT} RENAME copyright) +vcpkg_copy_pdbs() -vcpkg_fixup_cmake_targets(CONFIG_PATH "lib/cmake/GameNetworkingSockets" TARGET_PATH "share/GameNetworkingSockets") - +# Cleanup some file droppings that our cmakefile really should +# not be publishing file(REMOVE_RECURSE "${CURRENT_PACKAGES_DIR}/debug/share") file(REMOVE_RECURSE "${CURRENT_PACKAGES_DIR}/debug/include") - - -vcpkg_copy_pdbs() \ No newline at end of file diff --git a/vcpkg_ports/gamenetworkingsockets/vcpkg.json b/vcpkg_ports/gamenetworkingsockets/vcpkg.json new file mode 100644 index 0000000..fd45fd1 --- /dev/null +++ b/vcpkg_ports/gamenetworkingsockets/vcpkg.json @@ -0,0 +1,29 @@ +{ + "$schema": "https://raw.githubusercontent.com/microsoft/vcpkg/master/scripts/vcpkg.schema.json", + "name": "gamenetworkingsockets", + "version-string": "1.3.0", + "description": "A basic transport layer for games.", + "homepage": "https://github.com/ValveSoftware/GameNetworkingSockets", + "dependencies": [ "protobuf", "openssl" ], + "default-features": [ "openssl" ], + "features": { + "openssl": { + "description": "Use OpenSSL as the crypto provider. This is the default.", + "dependencies": [ "openssl" ] + }, + "libsodium": { + "description": "Use libsodium as the crypto provider.", + "dependencies": [ "libsodium" ] + }, + "webrtc": { + "description": "Compiles WebRTC support for P2P.", + "dependencies": [ "abseil" ] + }, + "examples": { + "description": "Build the examples." + }, + "tests": { + "description": "Build the tests." + } + } +} \ No newline at end of file