Failed to open nvenc codec generic error in an external library как исправить

Обновлено: 01.07.2024

I updated it to version 27.0.0 and could not start the stream with my previous encoder settings. An error appeared - Failed to open NVENC codec. I had to switch from NVEС to x264 Encoder, which increased the load on the CPU.

Steps to Reproduce

Windows 7 x64
OBS 27.0.0
Settings - Output - Streaming - Encoder - NVENC
.

Anything else we should know?

The text was updated successfully, but these errors were encountered:

vvo1tv commented Jun 1, 2021

Strange, but in the previous version of the OBS everything worked

notr1ch commented Jun 1, 2021

guihkx commented Jun 4, 2021

The GeForce GTX 670 does not have NVENC, so this is expected and should never have worked.

If it worked, it was probably experimental at best and might have been removed in a driver update. Windows 7 is also end of life, so you're really in unsupported territory here.

Are you sure about that?

I've been using (and still am) NVENC to record with OBS on my GTX 660 since forever. I'm on Linux, though.

The 27.0.0 update from OBS also broke NVENC for me, but I got it working again by switching Output Mode to Advanced and disabling "Psycho Visual Tuning" on the NVIDIA NVENC H.264 encoder.

Here's the full output of OBS 27.0.0 (Flatpak) not being able to use NVENC:

Perhaps there should be a way to detect if the GPU supports such features before automatically enabling it?

derrod commented Jun 4, 2021

While support for Kepler (at least consumer) will be dropped with the next major Nvidia driver version it does appear that they were a bit premature with removing Kepler cards from their list here.

notr1ch commented Jun 4, 2021

Reopening this as it does seem we have broken compatibility with the oldest GPUs. Even if NVIDIA no longer support them, if they used to work it should still work.

NVIDIA periodically drops older cards off of its support matrix for NVENC/NVDEC, even if they do have the required hardware. This makes it appear, at first glance to current information sources, that Kepler GPUs do not support NVENC, so we responded as such.

As for why @vvo1tv and @guihkx encountered this encoder failure, I'll attempt to clarify what happened on the OBS side of things. I hope you'll forgive me for this lengthy explanation. I'll provide a summary at the start for those who don't want to read all the details.

tl;dr: We recently changed something that exposed Psycho Visual Tuning to all users and it was enabled by default. It doesn't appear to work on Kepler GPUs, hence the encoder failure. Disabling Psycho Visual Tuning should allow you to use NVENC.

Psycho Visual Tuning (Psycho AQ) has been enabled by default in the texture-based NVENC implementation ("jim-nvenc" or "NVIDIA NVENC H.264 (new)") since we introduced the feature on Windows versions newer than Windows 7. We initially enabled the setting by default in the FFmpeg NVENC implementation. However, this caused some issues, so we disabled it temporarily. After we tied the setting to the texture-based implementation, we then re-enabled it "temporarily", and we no longer passed it to FFmpeg.

Perhaps there should be a way to detect if the GPU supports such features before automatically enabling it?

Incidentally, in the textured-based implementation, we do check if the GPU supports this feature before trying to enable it.

I recently took a look at adding this check to our FFmpeg NVENC implementation, but it required shuffling a fair bit of code around to be able to automatically detect this, and I didn't get it working. Someone else may have to tackle that. However, with the number of Kepler GPUs actively used in the wild compared to newer GPUs, and with NVIDIA expected to shift Kepler GPUs from having active driver support to LTS possibly by the end of this year, I can't say that it'll be high priority since users can disable the options causing the encoder failure.

Сбой вывода. Не удалось начать запись в OBS

Причины ошибки вывода потока в OBS

Поскольку запись или трансляция стримов потребляют большие ресурсы видеопамяти все ошибки нужно начинать с проверки видеокарты и ее работы. Естественно, первым делом стоит убедиться, что вы загрузили последнюю версию драйверов на официальном сайте и обновили. Для видеокарт от NVIDIA после запуска любой игры или приложения в трее есть иконка, которая позволяет увидеть какая видеокарта задействована в данный момент. При работе с графикой и записи видео или стримов разумеется должна отображаться внешняя видеокарта.

Второе – часто программа конфликтует с продуктами от Adobe запущенными на компьютере в данный момент(Photoshop, InDesign, Premiere Pro, Illustrator или Adobe XD). Сохраните всё, что делали, выключите программу и попробуйте снова запустить запись. Эта ошибка вызвана тем, что графические редакторы задействуют ресурсы видеокарт по максимуму, а если карточка не последних поколений, выделить дополнительную память на стрим или запись она просто не сможет.

Если обновление драйверов не помогло, посмотрим что пишут в логах. Закройте все лишние работающие программы. Далее в запущенной программе OBS:

Справка, Log файлы, посмотреть текущий журнал

Файл, настройки, вывод в программе OBS

Устанавливаем значение 0 в настройках GPU

Переключаем кодек с с NVENC H.264 на любой другой

Обновляем кодеки Video Codec x264 и Windows 10 Codec Pack

Выводы

I have attached the log file. No matter what settings I change or what I do, I get this same error using Nvenc in OBS Studio.

I am running 64 bit OBS Studio, I have tried running it as administrator. I have run it with nothing else open and tried to record my desktop.

I have already reinstalled OBS studio multiple times in multiple different locations.
I have reinstalled my GPU drivers after cleaning them out using DDU.
I have changed every setting around in the output tab while in advanced mode.

Please someone help me fix this. I want to stream/record using Nvenc and OBS studio.
I want to stream with OBS studio specifically as it has all my scenes set up already, and my new GPU is coming in and I'm sure Nvenc encoding will not work even when I swap out the GPU as it seems to be an OBS Studio specific error.

NOTE: Nvenc encoding works with Streamlabs OBS for me. This leads me to believe my install of OBS Studio is broken, but reinstalling it in different locations has done nothing to stop this error.

If any of you help me fix this error, I will be insanely grateful.
Thanks in advance.

Attachments

koala

Active Member

In Settings->Advanced->Video, set Color Format to NV12 and Color Range to Partial.

TheRealNoman

New Member

UPDATE | I fixed it

Changing the Color Format/Range didn't seem to fix it either.

I did a complete clean install of OBS Studio. I not only uninstalled it, but also deleted the folders of any installations of OBS Studio before reinstalling it, and this seemed to fix the problem.

@koala thanks for the reply

Jogger2

New Member

csmgj

New Member

UPDATE | I fixed it

Changing the Color Format/Range didn't seem to fix it either.

I did a complete clean install of OBS Studio. I not only uninstalled it, but also deleted the folders of any installations of OBS Studio before reinstalling it, and this seemed to fix the problem.

@koala thanks for the reply

After you reinstalled it, was "Encoding" still set to hardware? Before you reinstalled, would it work with "Encoding" set to "Software"?

dalenacio

New Member

DerkWehler

New Member

In Settings->Advanced->Video, set Color Format to NV12 and Color Range to Partial.

Mine is already set like this. I tried changing the Encoder to "Software (x264)" (though I would prefer to use hardware). Sometimes it works, sometimes it doesn't. I tried "Software (x264) low CPU usage. " too, which does the same.

It seems like with Software, it works when I click "start recording", but when I use my hot key, it gives me the error. Yep, recurring: hot key gives codec error, button works (on software encoding).

tensai

New Member

I also had this problem. On another website, I found this can be caused by trying to use the GPU for encoding when the GPU doesn't support that encoding. Click "Settings" then select "Output". Change "Encoding" to "Software" under "Streaming" and "Recording." I am running into these codec/encoder errors using nvenc and looking for a fix but I don't see "encoding" when I go to output?? running 25.0.8

tensai

New Member

Oh by software you mean x264, so using cpu to encode and avoiding nvenc altogether. Not really a solution though as some need to use nvenc since their cpu's can't handle the encoding. For me I have used nvenc without a problem for a long time so it's weird that this is happening all of a sudden.

Ken_SF

New Member

I had this problem on a seldom used Windows 7 Pro 64 Bit computer. The Settings-->output-->change to software encoding did not work for me. What did work: update my NVIDIA drivers. That was a problem too as I first had to install Java and subsequently the NVIDIA driver web page kept asking me to install over and over, even though I had the most current version.

I finally, opened the NVIDIA control panel, found the model of my current driver and manually downloaded the NVIDIA driver that seemed to be the best match. There was a compatibility check portion of the NVIDIA driver installation, so hopefully, that will protect someone if they select the wrong NVIDIA driver; I selected the right one.

After installing the NVIDIA drivers I rebooted, launched OBS (updated to most recent version today - 06/15/2020) and recording worked perfectly with hardware or software encoding selected in settings-->output.

I hope this helps someone else.

Thanks for the prior posts; they helped me figure out what to do.

domaxi198

New Member

braintalked

New Member

I also had this problem. On another website, I found this can be caused by trying to use the GPU for encoding when the GPU doesn't support that encoding. Click "Settings" then select "Output". Change "Encoding" to "Software" under "Streaming" and "Recording." I had the same problem and you helped me fix it! Many thanks :-)

Awoolyx

New Member

Some of you guys used the solution of going with CPU Encoding (Software), but this isn't really great if you have an NVIDIA GPU.

Here's my fix if you are looking for NVENC:
In Settings > Output ensure that only have EITHER Streaming OR Recording set to Hardware Encoding (NVENC). Set the other one that you don't want to CPU/Software (x264). NVIDIA does not let you use NVENC twice/parallel.

If you want to stream and record, I'd recommend using NVENC for Recording and Software for Streaming, as recording has a much higher bitrate than streaming.

Recent drivers allow up to three NVENC sessions. As recording has practically unlimited bitrate, it's best to use NVENC on streaming so it can make the most of the limited bitrate.

Rutopia

New Member

I found that for me, it was scaling the video. Apparently I had it scale to an 8k resolution. (I was messing about with different scenes). Setting scaling to my screen resolution allowed recording with NVENC to continue.

Scaling can be found in Settings -> Output. It's under both Stream and Record. (freely translated from Dutch ;))

Hope it helps anyone else in the future.

oxkos

New Member

I have attached the log file. No matter what settings I change or what I do, I get this same error using Nvenc in OBS Studio.

I am running 64 bit OBS Studio, I have tried running it as administrator. I have run it with nothing else open and tried to record my desktop.

NOTE: Nvenc encoding works with Streamlabs OBS for me. This leads me to believe my install of OBS Studio is broken, but reinstalling it in different locations has done nothing to stop this error.

If any of you help me fix this error, I will be insanely grateful.
Thanks in advance.

I have been through quite a few threads on this issue and decided to reach out for some help.

When trying to use nvenc in obs to record local video, I get the following error:

Starting the output failed. Please check the log for details.
Note: If you are using the NVENC or AMD encoders, make sure your video drivers are up to date.

10:30:12 PM.112: [NVENC encoder: 'recording_h264'] Failed to open NVENC codec: Generic error in an external library

Nothing really new here, many posts have had similar issues.

I have followed some direction from different posts, and compiled ffmpeg from source. Here is some useful outputs:

ffmpeg -encoders 2>/dev/null | grep nvenc
V. h264_nvenc NVIDIA NVENC H.264 encoder (codec h264)
V. nvenc NVIDIA NVENC H.264 encoder (codec h264)
V. nvenc_h264 NVIDIA NVENC H.264 encoder (codec h264)
V. nvenc_hevc NVIDIA NVENC hevc encoder (codec hevc)
V. hevc_nvenc NVIDIA NVENC hevc encoder (codec hevc)

ffmpeg
ffmpeg version N-93543-gbb5efd1 Copyright (c) 2000-2019 the FFmpeg developers
built with gcc 7 (Ubuntu 7.3.0-27ubuntu1

18.04)
configuration: --prefix=/home/patrick/ffmpeg_build --pkg-config-flags=--static --extra-cflags=-I/home/patrick/ffmpeg_build/include --extra-ldflags=-L/home/patrick/ffmpeg_build/lib --extra-libs='-lpthread -lm' --bindir=/home/patrick/bin --enable-gpl --enable-libaom --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-nonfree --enable-nvenc
libavutil 56. 26.100 / 56. 26.100
libavcodec 58. 48.101 / 58. 48.101
libavformat 58. 27.100 / 58. 27.100
libavdevice 58. 7.100 / 58. 7.100
libavfilter 7. 48.100 / 7. 48.100
libswscale 5. 4.100 / 5. 4.100
libswresample 3. 4.100 / 3. 4.100
libpostproc 55. 4.100 / 55. 4.100
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]. <[outfile options] outfile>.

From everything I can see, it looks like my system should support nvenc.

Thank you for any direction/advice.

OS: Linux Mint 19.1 Tessa x86_64
CPU: Intel i7-6700HQ (8) @ 3.500GHz
GPU: NVIDIA GeForce GTX 1060 Mobile 6GB
NVIDIA Driver Version: 418.56

I have attempted two OBS installations: one from obs website (version 23.0.0) and one from flatpak (23.0.2).

i have been trying to get OBS to use the 1660 super as the encoder with NVENC however, every time that I try I get the same message
.

Attachments

FerretBomb

Active Member

OBS is running on your 2080. OBS can only use NVENC on the card it is running on. Set your GPU to 0, as the error box indicates.

That said, NVENC is a separate part of your GPU die totally dedicated only to video encoding. NVENC will not use any rendering resources for encoding. Unlike AMD, nVidia's encoder is well-thought-out, and works well. The only tiny slice of GPU time that OBS will take up is for color conversion, scaling, and compositing of sources. None of which is handled by NVENC to begin with.

Take the 1660 out of your machine. It's pointless to have it in there, and what you're trying to do can't be done anyway (and would totally flood your PCIe bus sending the full frames from card to card, if the devs allowed it).

If anything, having the 1660 in there is actually actively making your performance worse. It's splitting your PCIe lane allocation to x8/x8, instead of letting your 2080 have a full x16 allocation (which would make it perform better with OBS.

Yes, people have tried it. No, it doesn't work, and even if it did, the gains would be near-zero while leaving the NVENC core in your 2080 sitting completely idle.

BBQBible

New Member

OBS is running on your 2080. OBS can only use NVENC on the card it is running on. Set your GPU to 0, as the error box indicates.

Yes, people have tried it. No, it doesn't work, and even if it did, the gains would be near-zero while leaving the NVENC core in your 2080 sitting completely idle.

Читайте также: