ISA Control via Environment Variables in NCNN

[inspireMeNow]

This patch introduces a new mechanism in NCNN to control the enabling or disabling of specific Instruction Set Architecture (ISA) features through environment variables. This change provides flexibility to users, allowing them to optimize according to particular requirements or limitations.

Environment Variable Format

• Use the NCNN_ISA environment variable to set the instruction sets you want to disable.
• Separate each instruction set flag with comma.
• Prefix an instruction set flag with - to disable the corresponding instruction set.
• If no specific flags are provided, all supported instruction sets are enabled by default.

Supported Instruction Set Flags

Depending on the architecture, the following instruction set flags are available (partial list):

• ARM Architecture: -cpuid, -asimdhp, -asimddp, -neon, -bf16, -sve, -sve2, -i8mm
• x86 Architecture: -avx, -fma, -xop, -f16c, -avx2, -avx512, -avx_vnni, -avx512_vnni, -avx512_bf16, -avx512_fp16
• Loongarch Architecture: -lsx, -lasx
• MIPS Architecture: -msa
• Loongson Architecture: -mmi
• RISC-V Architecture: -rvv, -zfh

Changes Introduced

1. get_isa_env Function:
   • A new function get_isa_env is introduced to parse the environment variable NCNN_ISA. This function reads the ISA flags passed as a string and checks whether specific CPU features should be disabled.
   • If the environment variable contains a feature preceded by a -, such as -neon, the feature is disabled, and a warning is printed.
   • The function uses strdup() to make a copy of the NCNN_ISA string and processes it using strtok(), breaking it down into tokens and comparing each against the provided ISA feature.
2. Global variables:
   • I made is_cpu_x86_avx_disabled a global variable, allowing the cpu_support_x86_avx function to directly reference it to modify its return value. By doing this, it avoid repeatedly parsing the environment variable in every call to the function. The global variable is initialized once, based on the environment variable NCNN_ISA. Thus it ensures that the environment setting is only parsed once.
3. Architecture-Specific:
   • I also use preprocessor macros, such as __mips__, to determine which global variables are relevant based on the target architecture. Thus we conditionally compile code that checks the appropriate global variables for features supported by that architecture. For example, if __mips__ is defined, the code will use the is_cpu_mips_msa_disabled global variable to check whether MSA support is disabled.

Usage Examples

Here is an example for x86 linux, you can also use NCNN_ISA on other architectures, such as ARM, RISC-V.

Install required build dependencies

On Debian 10+, Ubuntu 20.04+

sudo apt install build-essential git cmake libprotobuf-dev protobuf-compiler libomp-dev libvulkan-dev vulkan-tools libopencv-dev

On Redhat based distribution

sudo yum install build-essential git cmake libprotobuf-dev protobuf-compiler libvulkan-dev vulkan-utils libopencv-dev

Build and install the project

mkdir build-x86
cd build-x86
cmake -DCMAKE_TOOLCHAIN_FILE=../toolchains/host-c.gcc.toolchain.cmake -DNCNN_BUILD_TESTS=ON ..
make -j$(nproc)
make install 

Here is an example program for reference

#include <ncnn/cpu.h>
#include <iostream>

int main()
{
    printf("avx support: %s\n", ncnn::cpu_support_x86_avx()? "true" : "false");
    printf("avx512 support: %s\n", ncnn::cpu_support_x86_avx512()? "true" : "false");
    printf("fma support: %s\n", ncnn::cpu_support_x86_fma()? "true" : "false");
    printf("xop support: %s\n", ncnn::cpu_support_x86_xop()? "true" : "false");
    printf("f16c support: %s\n", ncnn::cpu_support_x86_f16c()? "true" : "false");
    printf("avx2 support: %s\n", ncnn::cpu_support_x86_avx2()? "true" : "false");
    printf("avx_vnni support: %s\n", ncnn::cpu_support_x86_avx_vnni()? "true" : "false");
    printf("avx512_vnni support: %s\n", ncnn::cpu_support_x86_avx512_vnni()? "true" : "false");
    printf("avx512_bf16 support: %s\n", ncnn::cpu_support_x86_avx512_bf16()? "true" : "false");
    printf("avx512_fp16 support: %s\n", ncnn::cpu_support_x86_avx512_fp16()? "true" : "false");
    return 0;
}

Compile the example program.

g++ -o test_isa test_isa.cpp -Iinclude -Llib64 -lncnn -fopenmp

Test the example program.

Normal

user@fedora:~/Github/Mine/ncnn/ncnn/build-x64/install$ ./test_isa
avx support: true
avx512 support: true
fma support: true
xop support: false
f16c support: true
avx2 support: true
avx_vnni support: false
avx512_vnni support: true
avx512_bf16 support: false
avx512_fp16 support: false

Disable AVX512

user@fedora:~/Github/Mine/ncnn/ncnn/build-x64/install$ NCNN_ISA=-avx512 ./test_isa
warning: avx512 disabled via environment variable!
avx support: true
avx512 support: false
fma support: true
xop support: false
f16c support: true
avx2 support: true
avx_vnni support: false
avx512_vnni support: false
avx512_bf16 support: false
avx512_fp16 support: false