Note: The patch in this post is out-dated because pipewire has already added the support in this commit. The commit message describes the exact case we need that support: encode a screencast with a hardware encoder.
The issue
When constructing a gstreamer pipeline for RTSP, we use pipewiresrc element to get the video stream from Weston and vpuenc_h264 element to encode it to H.264 format with VPU before streaming out to rtph264pay element. But between pipewiresrc and vpuenc_h264, we have to explicitly specify the resolution of the video:
pipewiresrc ! video/x-raw,format=BGRx,width=800,height=480 ! vpuenc_h264 ! rtph264pay name=pay0 pt=96
After enabling the debug log of GStreamer, I’ve figured out that during the negotiation of capabilities, the size in capabilities of pipewiresrc is determined values 800x480 (type SPA_CHOICE_None in pipewire), but the size in capabilities of vpuenc_h264 is a step values [64, 1920, 8] and [64, 1088, 8] (type SPA_CHOICE_Step in pipewire).
However, pipewire doesn’t support comparing (filtering) SPA_CHOICE_None with SPA_CHOICE_Step or vice versa, so it will fail to negotiate the capabilities if pipewiresrc and vpuenc_h264 is connected directly without explicitly specify the resolution inbetween.
The fix
The fix is obvious: add the support of filtering step values in pipewire.
SPA_CHOICE_Step is pretty much similar to SPA_CHOICE_Range so could deal them in the same way. So the patch is as simple as below (also add support of comparing with SPA_CHOICE_Enum to keep consistency):
diff --git a/spa/include/spa/pod/filter.h b/spa/include/spa/pod/filter.h
index dd2e9931..f2c996be 100644
--- a/spa/include/spa/pod/filter.h
+++ b/spa/include/spa/pod/filter.h
@@ -153,7 +153,9 @@ spa_pod_filter_prop(struct spa_pod_builder *b,
}
if ((p1c == SPA_CHOICE_None && p2c == SPA_CHOICE_Range) ||
- (p1c == SPA_CHOICE_Enum && p2c == SPA_CHOICE_Range)) {
+ (p1c == SPA_CHOICE_Enum && p2c == SPA_CHOICE_Range) ||
+ (p1c == SPA_CHOICE_None && p2c == SPA_CHOICE_Step) ||
+ (p1c == SPA_CHOICE_Enum && p2c == SPA_CHOICE_Step)) {
int n_copied = 0;
/* copy all values inside the range */
for (j = 0, a1 = alt1, a2 = alt2; j < nalt1; j++, a1 = SPA_MEMBER(a1,size,void)) {
@@ -169,13 +171,10 @@ spa_pod_filter_prop(struct spa_pod_builder *b,
nc->body.type = SPA_CHOICE_Enum;
}
- if ((p1c == SPA_CHOICE_None && p2c == SPA_CHOICE_Step) ||
- (p1c == SPA_CHOICE_Enum && p2c == SPA_CHOICE_Step)) {
- return -ENOTSUP;
- }
-
if ((p1c == SPA_CHOICE_Range && p2c == SPA_CHOICE_None) ||
- (p1c == SPA_CHOICE_Range && p2c == SPA_CHOICE_Enum)) {
+ (p1c == SPA_CHOICE_Range && p2c == SPA_CHOICE_Enum) ||
+ (p1c == SPA_CHOICE_Step && p2c == SPA_CHOICE_None) ||
+ (p1c == SPA_CHOICE_Step && p2c == SPA_CHOICE_Enum)) {
int n_copied = 0;
/* copy all values inside the range */
for (k = 0, a1 = alt1, a2 = alt2; k < nalt2; k++, a2 = SPA_MEMBER(a2,size,void)) {
@@ -220,15 +219,11 @@ spa_pod_filter_prop(struct spa_pod_builder *b,
if (p1c == SPA_CHOICE_Enum && p2c == SPA_CHOICE_Flags)
return -ENOTSUP;
- if (p1c == SPA_CHOICE_Step && p2c == SPA_CHOICE_None)
- return -ENOTSUP;
if (p1c == SPA_CHOICE_Step && p2c == SPA_CHOICE_Range)
return -ENOTSUP;
if (p1c == SPA_CHOICE_Step && p2c == SPA_CHOICE_Step)
return -ENOTSUP;
- if (p1c == SPA_CHOICE_Step && p2c == SPA_CHOICE_Enum)
- return -ENOTSUP;
if (p1c == SPA_CHOICE_Step && p2c == SPA_CHOICE_Flags)
return -ENOTSUP;
After applying this patch, the pipeline for RTSP can be simplified to:
pipewiresrc ! vpuenc_h264 ! rtph264pay name=pay0 pt=96
which is screen resolution agnostic and can be used generally on different systems.
Have fun!