Avaudioplayer

AVFoundationのAVVideoComposition



Avvideocomposition Avfoundation



ビデオのカスタム再生を実現するには、ビデオリソースをまとめるだけでは不十分であり、AVMutableVideoCompositionを使用する必要があります。クラスは、さまざまなビデオアセットがさまざまな時間枠でどのように再生されるかを定義します。

AVVideoComposition

AVVideoCompositionはいAVMutableVideoComposition親クラス、その主なプロパティおよびメソッドは次のとおりです。



/ / The class of this class, when providing a custom constructor class, the provided class must comply with the AVVideoCompositing protocol @property (nonatomic, readonly, nullable) Class customVideoCompositorClass NS_AVAILABLE(10_9, 7_0) / / Video refresh time per frame @property (nonatomic, readonly) CMTime frameDuration / / size range when the video is displayed @property (nonatomic, readonly) CGSize renderSize //The scale of the video display range size (only valid for iOS) @property (nonatomic, readonly) float renderScale / / Describe the collection of specific video playback mode information in the video collection, which is a class instance object that follows the AVVideoCompositionInstruction protocol / / These video playback information constitutes a complete timeline, can not overlap, can not be interrupted, and the order in the array is the playback order of the corresponding video @property (nonatomic, readonly, copy) NSArray *instructions / / Tool object for combining Core Animation of video frames and dynamic layers, can be nil @property (nonatomic, readonly, retain, nullable) AVVideoCompositionCoreAnimationTool *animationTool //Create an instance directly with an asset. The properties of the created instance are evaluated and adapted according to the properties of all the video tracks in the asset, so before calling this method, make sure the properties in the asset are loaded. / / The object of the returned instance object the instructions in the instructions will correspond to the attribute requirements in the track in each asset //The property of the returned instance object The value of frameDuration is the value of the nominalFrameRate property of all tracks in the asset. If these values ​​are 0, the default is 30fps. //The property of the returned instance object The value of renderSize is the value of the asset's naturalSize property if asset is an instance of the AVComposition class. Otherwise, the value of renderSize will contain the value of the naturalSize property for each track + (AVVideoComposition *)videoCompositionWithPropertiesOfAsset:(AVAsset *)asset NS_AVAILABLE(10_9, 6_0) //These three properties set the color space, matrix, and color conversion function when rendering the frame. Possible values ​​are defined in the AVVideoSetting.h file. @property (nonatomic, readonly, nullable) NSString *colorPrimaries NS_AVAILABLE(10_12, 10_0) @property (nonatomic, readonly, nullable) NSString *colorYCbCrMatrix NS_AVAILABLE(10_12, 10_0) @property (nonatomic, readonly, nullable) NSString *colorTransferFunction NS_AVAILABLE(10_12, 10_0) //This method returns an instance that specifies the block to render each frame of each valid track in the asset to get the CIImage instance object. / / Render each frame in the block, after the success should call the request method finishWithImage: context: and the resulting CIImage object as a parameter / / If the rendering fails, you should call the finishWithError: method and pass the error message + (AVVideoComposition *)videoCompositionWithAsset:(AVAsset *)asset applyingCIFiltersWithHandler:(void (^)(AVAsynchronousCIImageFilteringRequest *request))applier NS_AVAILABLE(10_11, 9_0)

AVMutableVideoComposition

AVMutableVideoCompositionはいAVVideoComposition親クラスのプロパティを継承する変数サブクラスを変更することができ、以下の作成メソッドが追加されました。

//The value of the property of the instance object created by this method is nil or 0, but its properties are all modifiable. + (AVMutableVideoComposition *)videoComposition

AVVideoCompositionInstruction

上記の2つのクラスでは、実際にビデオ再生モードを含む情報は命令属性であり、このコレクション内のオブジェクトが追跡されます。AVVideoCompositionInstructionプロトコル、カスタムクラスを使用しない場合は、AVFoundationフレームワークを使用できますAVVideoCompositionInstructionクラス。
このクラスの関連属性は次のとおりです。



// indicates the time range in which the instruction takes effect @property (nonatomic, readonly) CMTimeRange timeRange / / specify the background color of the composition of the current time period / / If not specified, then use the default black //If the rendered pixel has no transparency channel, then this color will also ignore transparency. @property (nonatomic, readonly, retain, nullable) __attribute__((NSObject)) CGColorRef backgroundColor //AVVideoCompositionLayerInstruction class collection of instance objects describing the hierarchy and composition of each video resource frame / / In the order of this array, the first one is displayed on the first layer, the second one is displayed below the first layer, and so on. @property (nonatomic, readonly, copy) NSArray *layerInstructions / / Indicates whether the video frame of the time period needs post processing / / If NO, the processing of the later layer will skip this time period, which can improve efficiency // YES is processed by default (refer to the AVVideoCompositionCoreAnimationTool class) @property (nonatomic, readonly) BOOL enablePostProcessing //The set of all track IDs in the current instruction that need to be combined with the frame, calculated by the attribute layerInstructions @property (nonatomic, readonly) NSArray *requiredSourceTrackIDs NS_AVAILABLE(10_9, 7_0) / / If the current instruction after the video frame combination in the time period, the actual get a frame of a source video, then return the ID of the video resource @property (nonatomic, readonly) CMPersistentTrackID passthroughTrackID NS_AVAILABLE(10_9, 7_0)

AVMutableVideoCompositionInstruction

AVMutableVideoCompositionInstructionはいAVVideoCompositionInstruction継承された親クラスのプロパティを変更できるサブクラスであり、プロパティ値がnilまたは無効のインスタンスを作成するためのメソッドを提供します。

+ (instancetype)videoCompositionInstruction

AVVideoCompositionLayerInstruction

AVVideoCompositionLayerInstructionこれは、特定のビデオリソースのさまざまな再生モードを説明するクラスです。以下の方法を使用して、アフィン変化、透明度変化、および作物領域変化の勾配情報を取得できます。

/ / Get the affine gradient information containing the specified time //startTransform, endTransform are used to receive the start and end values ​​of the change process //timeRange is used to receive the duration of the change / / The return value indicates whether the specified time time is within the change time timeRange - (BOOL)getTransformRampForTime:(CMTime)time startTransform:(nullable CGAffineTransform *)startTransform endTransform:(nullable CGAffineTransform *)endTransform timeRange:(nullable CMTimeRange *)timeRange / / Get the transparency gradient information containing the specified time //startOpacity, endOpacity is used to receive the start and end values ​​of the transparency change process //timeRange is used to receive the duration of the change / / The return value indicates whether the specified time time is within the change time timeRange - (BOOL)getOpacityRampForTime:(CMTime)time startOpacity:(nullable float *)startOpacity endOpacity:(nullable float *)endOpacity timeRange:(nullable CMTimeRange *)timeRange / / Get the gradient information of the crop area containing the specified time //startCropRectangle, endCropRectangle are used to receive the start and end values ​​of the change process //timeRange is used to receive the duration of the change / / The return value indicates whether the specified time time is within the change time timeRange - (BOOL)getCropRectangleRampForTime:(CMTime)time startCropRectangle:(nullable CGRect *)startCropRectangle endCropRectangle:(nullable CGRect *)endCropRectangle timeRange:(nullable CMTimeRange *)timeRange NS_AVAILABLE(10_9, 7_0)

AVMutableVideoCompositionLayerInstruction

AVMutableVideoCompositionLayerInstructionはいAVVideoCompositionLayerInstructionコンポジション内のトラックリソースのアフィン変更、クリッピング領域、透明度などを変更するサブクラス。



このサブクラスは、親クラスと比較してインスタンスを作成する方法も提供します。

//The difference between the two methods is that the value of the trackID of the instance object returned by the former is the trackID value of the track. //The value of the trackID of the instance object returned by the second method is kCMPersistentTrackID_Invalid + (instancetype)videoCompositionLayerInstructionWithAssetTrack:(AVAssetTrack *)track + (instancetype)videoCompositionLayerInstruction

このクラスの属性は、命令が作用するトラックのIDを表します。

@property (nonatomic, assign) CMPersistentTrackID trackID

trackIDを設定した後、次の方法でシェービング情報を設定します。

/ / Set the affine change information of the frame in the video / / specifies the time range, start value and end value of the change, where the origin of the coordinate system is the upper left corner, the downward direction is the positive direction - (void)setTransformRampFromStartTransform:(CGAffineTransform)startTransform toEndTransform:(CGAffineTransform)endTransform timeRange:(CMTimeRange)timeRange / / Set the affine transformation of the specified time in the timeRange range of the instruction, the value will remain until it is set again - (void)setTransform:(CGAffineTransform)transform atTime:(CMTime)time / / Set the gradient information of the transparency, the initial value and end value of the transparency should be between 0 and 1. //The process of change is linear - (void)setOpacityRampFromStartOpacity:(float)startOpacity toEndOpacity:(float)endOpacity timeRange:(CMTimeRange)timeRange / / Set the transparency of the specified time, the transparency will continue until the next value is set - (void)setOpacity:(float)opacity atTime:(CMTime)time / / Set the change information of the crop rectangle - (void)setCropRectangleRampFromStartCropRectangle:(CGRect)startCropRectangle toEndCropRectangle:(CGRect)endCropRectangle timeRange:(CMTimeRange)timeRange NS_AVAILABLE(10_9, 7_0) / / Set the clipping rectangle of the specified time - (void)setCropRectangle:(CGRect)cropRectangle atTime:(CMTime)time NS_AVAILABLE(10_9, 7_0)

AVVideoCompositionCoreAnimationTool

ビデオ再生をカスタマイズする場合、透かし、タイトル、またはその他のアニメーション効果を追加する必要がある場合があります。このクラスを使用する必要があります。このクラスは通常、オフラインビデオのレイヤーとアニメーションレイヤーの組み合わせを調整するために使用されます（使用されている場合）AVAssetExportSession AVAssetReaderで、AVAssetReaderクラスがビデオファイルをエクスポートまたはビデオファイルを読み取るとき、それがオンラインリアルタイムビデオ再生である場合は、AVSynchronizedLayerクラスを使用して、ビデオの再生とアニメーション効果を同期する必要があります。

このクラスを使用する場合、動画全体のタイムラインでアニメーションを変更できるため、アニメーションの開始時刻をAVCoreAnimationBeginTimeAtZeroに設定する必要があることに注意してください。 、この値は実際には0より大きい、属性値removedOnCompletionアニメーションの実行が終了したときに削除されないように、NOに設定する必要があります。また、UIViewに関連付けられたレイヤーを使用しないでください。

ビデオの組み合わせの後処理ツールクラスとしての主なメソッドは次のとおりです。

//AVVideoCompositionInstruction property layerInstructions contains the AVVideoCompositionLayerInstruction instance object that should have // The trackID is consistent with the AVVideoCompositionLayerInstruction instance object, and for performance reasons, this object should not be used to set transform changes //In iOS, CALayer is the background layer of UIView, and its content can be flipped, which is determined by the method contentsAreFlipped (if all layers include sublayers, the number returned by this method is YES, the number is YES, Indicates that the content in the layer can be flipped vertically) //So if the layer here is used to set the layer property of UIView, or as a sublayer of it, its property value geometryFlipped should be set to YES , so that it can keep the result of whether it can be flipped. + (instancetype)videoCompositionCoreAnimationToolWithAdditionalLayer:(CALayer *)layer asTrackID:(CMPersistentTrackID)trackID // Render the combined video frame placed in the layer videoLayer along with the content in the animation layer's animationLayer to get the final video frame. //Normally, videoLayer is a sublayer of animationLayer , and animationLayer is not in any layer tree + (instancetype)videoCompositionCoreAnimationToolWithPostProcessingAsVideoLayer:(CALayer *)videoLayer inLayer:(CALayer *)animationLayer //Copy every layer in videoLayers and render it with the animationLayer to get the most frame //// Normally, the layers in videoLayers are in the layer tree of the animationLayer, and the animationLayer does not belong to any layer tree. + (instancetype)videoCompositionCoreAnimationToolWithPostProcessingAsVideoLayers:(NSArray *)videoLayers inLayer:(CALayer *)animationLayer NS_AVAILABLE(10_9, 7_0)

AVVideoCompositionValidationHandling

編集後にビデオリソースアセットを取得し、リソースのカスタム再生情報ビデオコンポジションを設定する場合、ビデオコンポジションがこのアセットに対して有効かどうかを確認する必要があります。AVVideoComposition検証方法。

/* @param asset Set the checksum of the first parameter, set nil to ignore these checks 1. This method can check if the attribute instructions of `AVVideoComposition` meet the requirements. 2. Verify in the layerInstructions property of each `AVVideoCompositionInstruction` object in the instructions Whether each `AVVideoCompositionLayerInstruction` object trackID value corresponds to the ID of the track in the asset Or an animationTool instance of `AVVideoComposition` 3. Verify time The length of asset is contrary to the time range in instructions @param timeRange Set the checksum content of the second parameter 1. Verify that all time ranges of instructions are within the range of timeRange provided, To ignore this check, pass the parameter `CMTimeRangeMake(kCMTimeZero, kCMTimePositiveInfinity)` @param validationDelegate Set the proxy class that follows the AVVideoCompositionValidationHandling protocol to handle errors in the validation process, which can be nil */ - (BOOL)isValidForAsset:(nullable AVAsset *)asset timeRange:(CMTimeRange)timeRange validationDelegate:(nullable id)validationDelegate NS_AVAILABLE(10_8, 5_0)

合意に従うようにプロキシオブジェクトを設定しますAVVideoCompositionValidationHandlingオブジェクトが次のプロトコルメソッドを実装している場合、渡された構成パラメーターが変更されると、上記の検証メソッドは例外をスローします。

プロトコルは次のコールバックメソッドを提供し、すべてのメソッドの戻り値を使用して、さらにエラーを取得するためにチェックを続行するかどうかを決定します。

//Report an invalid value in videoComposition - (BOOL)videoComposition:(AVVideoComposition *)videoComposition shouldContinueValidatingAfterFindingInvalidValueForKey:(NSString *)key NS_AVAILABLE(10_8, 5_0) / / Report there is no corresponding instruction in the videoComposition time period - (BOOL)videoComposition:(AVVideoComposition *)videoComposition shouldContinueValidatingAfterFindingEmptyTimeRange:(CMTimeRange)timeRange NS_AVAILABLE(10_8, 5_0) / / Report instructions in videoComposition timeRange invalid instance object //may be timeRange itself is CMTIMERANGE_IS_INVALID // or the time period overlaps with the timeRange of the previous instruction //It may be that the start time is earlier than the start time of the previous instruction's timeRange - (BOOL)videoComposition:(AVVideoComposition *)videoComposition shouldContinueValidatingAfterFindingInvalidTimeRangeInInstruction:(id)videoCompositionInstruction NS_AVAILABLE(10_8, 5_0) //Report the layer instruction in videoComposition is inconsistent with the trackID of the track in the asset specified when the validation method is called. //Do not match the trackID of the animationTool used by composition - (BOOL)videoComposition:(AVVideoComposition *)videoComposition shouldContinueValidatingAfterFindingInvalidTrackIDInInstruction:(id)videoCompositionInstruction layerInstruction:(AVVideoCompositionLayerInstruction *)layerInstruction asset:(AVAsset *)asset NS_AVAILABLE(10_8, 5_0)