Difference between revisions of "Data-driven Semantic Indexing of Time-based Broadcast Media Series"
(→Method) |
(→Method) |
||
| Line 9: | Line 9: | ||
===Method=== | ===Method=== | ||
| − | We are investigating the incorporation of data generated with computational methods into a semantic indexing method for a series of documentary episodes post-digitization using AI/ML | + | We are investigating the incorporation of data generated with computational methods into a semantic indexing method for a series of documentary episodes post-digitization using AI/ML processes and using “deep hyperlinks” as locators that directly point within digital video. Our research question was "Can named entities and features detected by AI/ML processes serve as a data source for the data-driven semantic indexing of documentary series?" |
A preliminary evaluation of this question led us to use Microsoft [https://docs.microsoft.com/en-us/azure/azure-video-analyzer/ Azure Video Analyzer] service to generate JSON files with various [https://docs.microsoft.com/en-us/azure/azure-video-analyzer/video-analyzer-for-media-docs/video-indexer-overview#video-insights detected features]. For this proof of concept, we concentrated on just a few of the detected features (what MS refers to as “insights”): Transcripts lines and transcript blocks as well as detected topics and recognized named entities (persons and locations). We extracted these features from the JSON output, transformed them into RDF triples, and loaded them into our local Wikibase instance for SPARQL querying. | A preliminary evaluation of this question led us to use Microsoft [https://docs.microsoft.com/en-us/azure/azure-video-analyzer/ Azure Video Analyzer] service to generate JSON files with various [https://docs.microsoft.com/en-us/azure/azure-video-analyzer/video-analyzer-for-media-docs/video-indexer-overview#video-insights detected features]. For this proof of concept, we concentrated on just a few of the detected features (what MS refers to as “insights”): Transcripts lines and transcript blocks as well as detected topics and recognized named entities (persons and locations). We extracted these features from the JSON output, transformed them into RDF triples, and loaded them into our local Wikibase instance for SPARQL querying. | ||
Revision as of 22:58, 15 June 2021
Introduction
In this project, the Linked Data Research Group at UA SLIS seeks to generalize our semantic indexing method for sports image and video clip collections to collections ("series") of documentaries and other broadcast genres.
Our goal is to investigate models for the semantic indexing of broadcast series for granular access and semantic integration across the linked data cloud using data generated by ML/AI computational methods applied to the documentaries of the series.
At this time, we can report very preliminary results and example SPARQL queries over 5 AABP-hosted documentaries in The Alabama Experience series with data generated by the Microsoft Azure Video Analyzer service.
PLEASE NOTE: This is a proof of concept demo, and we acknowledge some issues with our semantic data model and with SPARQL query construction.
Method
We are investigating the incorporation of data generated with computational methods into a semantic indexing method for a series of documentary episodes post-digitization using AI/ML processes and using “deep hyperlinks” as locators that directly point within digital video. Our research question was "Can named entities and features detected by AI/ML processes serve as a data source for the data-driven semantic indexing of documentary series?"
A preliminary evaluation of this question led us to use Microsoft Azure Video Analyzer service to generate JSON files with various detected features. For this proof of concept, we concentrated on just a few of the detected features (what MS refers to as “insights”): Transcripts lines and transcript blocks as well as detected topics and recognized named entities (persons and locations). We extracted these features from the JSON output, transformed them into RDF triples, and loaded them into our local Wikibase instance for SPARQL querying.
We assembled a sub-collection of 5 documentaries from The Alabama Experience series all from the same year (1997):
- Roses of Crimson (60 minutes)
- Natural Assets (30 minutes)
- Miller’s Pottery (30 minutes)
- High Calling (30 minutes)
- A Season with the Forgotten Farmers (30 minutes)
