Heavy Reconstruction Open Source Let H5 Label Replace C++ to Decode and Play Audio File of speex Compression Protocol in Real Time [IM Gospel]

  c++, html5, javascript, node.js, typescript

图片描述

Such a cow force wheel, must be beautiful figure town building

Speex is an open source, free and patent-free audio compression format mainly for voice.

This wheel is suitable for super-large projects, because the library itself is very large, of courseIMThere is no small project for the project, is there?

  • Speex (phonetic symbol [spi:ks])It is an open source, free, patent-free audio compression format designed for voice.SpeexThe project aims to reduce the entry threshold for voice applications by providing free alternatives to expensive dedicated voice codecs. In addition,SpeexIt is very suitable for Internet applications and provides useful features that do not exist in most other codecs. Finally,Speex is GNUPart of the project can be revisedBSDUse under license.

Coding process

  • UseSpeexTheAPIFunction to compress and encode audio data goes through the following steps:
  • Define aSpeexBitsType variablebitsAnd oneSpeexMemory pointer variable of encoderenc.
  • callspeex_bits_init(&bits)Function initializationbits.
  • callenc = speex_encoder_init(&speex_nb_mode)Function initializationenc. among themspeex_nb_modeYesSpeexModeA variable of type that represents a narrow-band pattern. in additionspeex_wb_modeSaid broadband mode,speex_uwb_modeIndicates ultra-wideband mode.
  • Calling functionint speex_encoder_ctl(void * state, int request, void * ptr)To set parameters of the encoder, wherein the parameter state represents a memory pointer of the encoder; The parameter request indicates the type of parameter to be defined, such asSPEEX_GET_FRAME_SIZEIndicates to set the frame size.SPEEX_SET_QUALITYIndicates the quality level of the code; The parameter ptr represents the value to be set.
  • After initialization is completed, each frame of sound is processed as follows: call the functionspeex_bits_reset(&bits)ResetbitsAnd then call the functionspeex_encode(enc_state,input_frame, &bits)After encoding, the encoded is saved in the parameter bitsSpeexFormat data frame.

After encoding, call the functionspeex_bits_destroy(&bits),speex_encoder_destroy(enc_state)To destroySpeexBitsAnd encoders.

Decoding flow

  • For those already codedSpeexThe decoding of the format audio data frame goes through the following steps:
  • Define aSpeexBitsType variablebitsAnd oneSpeexMemory pointer variable of decoderdec.
  • callspeex_bits_init(&bits)Function initializationbits.
  • calldec = speex_decoder_init(&speex_nb_mode)Function initializationdec.
  • Calling functionspeex_decoder_ctl(void * state, int request, void * ptr)To set the decoder parameters.
  • Calling functionspeex_decode(void * state, SpeexBits * bits, float * out)For theSpeexThe format audio data frame is decoded, and the decoded audio data frame is stored in the parameter out.
  • Calling functionspeex_bits_destroy(&bits), speex_decoder_destroy(void * state)To destroySpeexBitsAnd decoder

Point out

As instant messaging products, mobile phones such as WeChat are likely to be acceptedspeexAudio file after protocol compression. Of course, the file suffix iswavOr ..oggIt doesn’t matter

  • H5TheaudioLabels can be played

    • Audio Format and Browser Support
    • At present,<audio>Element supports three audio format files:MP3, Wav, and Ogg:
    • BrowserMP3 Wav Ogg
    • Internet Explorer 9+ YES NO NO
    • Chrome 6+ YES YES YES
    • Firefox 3.6+ NO YES YES
    • Safari 5+ YES YES NO
    • Opera 10+ NO YES YES
    • Audio formatMIMEType
    • Format MIME-type
    • MP3 audio/mpeg
    • Ogg audio/ogg
    • Wav audio/wav

This open source library is based onspeexEncapsulation, after extracting necessary files, further encapsulates and modifies compatibility in complex environment

  • This source code supports the environment

    • NativejavaScriptTheHTMLEnvironment
    • MVVMFramework
    • Electron React dva webpackCross-platform complex environment based on

Special warning: this source code does not supportAMD CMD commonJS ES6As well as any modularization scheme, can only be used by calling functions after the script tag is introduced.

The invention mainly solves the problem that speex audio format files in instant messaging are directly played in H5

  • This project must be run in a server environment and cannot be used by opening index.html files locally (because websocket communication is used)
  • Audio files in Speex format can be played directly through the Audio tab on H5 pages.
  • Can be in complex environment, such asElectron + webpack +dva + ReactPerfect for cross-platform use of
  • For data with a frequency less than 22khz, we need to copy it and simulate it to 22khz, because H5 only supports data with a frequency greater than 22khz, but this kind of situation is not commonly encountered by us.
  • Use the < script > tag to introduce the source code
  • If you use < audio src = “../test.ogg” > directly in the page, please call the function initAudio () at the top of the global JS code immediately after introducing the source code
  • If you use the audio tag dynamically or modify the src attribute of the audio tag dynamically, please call the function initAudio () after modifying the src attribute each time, otherwise you cannot play audio files in speex format.
  • Speex format audio files, suffix may be ogg, but can still be played (speex is only an open source free compression protocol)
  • This project does not support any modularization. import or require is prohibited in the framework, because the source code involves octal code, which cannot be used in strict mode.
  • MVVM framework and native JS usage methods are both introduced through script, and initAudio function is called globally

WelcomegithubSubmitissueThis wheel is optimized after integrating other people’s code and will continue to be optimized later.Github warehouse address

npm i speex-in-h5It can also be downloaded, but this project does not support any modularization scheme, please introduce it manually.indexDocuments, in the warehouseDemo!

Passing by to praise,6 days 6 times 9———996The good news, teacher ma