Developer Documentation: Custom Metadata Fields

[stuckless]

BMT uses a number of custom metadata fields. Here is a list of all the fields that BMT uses. This list is provided so that other developers can use these fields when populating metadata, instead of creating new fields.

For now, most of the documentation is empty, but I'll try to fill in the blanks over time. This java file will be updated first with any new custom metadata fields, so the java file, is the best place to look.


Custom Metadata Fields

MediaTitle - Movie Title or Series Title - Used for Fanart Lookup

MediaType - TV, Movie, Music - The type of media file - Used for Fanart Lookup

SeasonNumber - Season Number from TVDB (cannot be 0)

EpisodeNumber - Episode Number from TVDB (cannot be 0)

IMDBID - imdb ID from imdb.com

DiscNumber - For multi-disc movies, this is the disc number

MediaProviderID - Metadata Provider used for fetching information (ie, imdb, tmdb, tvdb, dvdprofiler, etc)

MediaProviderDataID - The Unique data id for this media file as known by the Metadata Provider

UserRating - User's Rating at the time the mediafile was scraped. A value between 0 and 100, so a rating of "9.5" would actually be "95"

TrailerUrl - Trailer URL for this media file

SeriesInfoID - SageTV's SeriesInfo id for this media file

DefaultPoster - Path of the default poster relative to the Phoenix CentralFanart dir

DefaultBanner - Path of the default banner relative to the Phoenix CentralFanart dir

DefaultBackground - Path of the default background relative to the Phoenix CentralFanart dir

[PLUCKYHD]

Nice info any chance and I know it is a big request but

DefaultPoster - Path of the default poster relative to the Phoenix CentralFanart dir

DefaultBanner - Path of the default banner relative to the Phoenix CentralFanart dir

DefaultBackground - Path of the default background relative to the Phoenix CentralFanart dir

any chance
Those the could not become not relative to the central fanart directory or we add another field to say if it is. That would allow those of us working with other providers to store fanart accessible to phoenix that don't follow the phoenix fanart structure. That is assuming your Fanart class pulls from those fields.

Just a request if not possible I understand but have been considering using the sagedb for some future things and just thinking ahead a bit.

Skye

[stuckless]

Originally Posted by PLUCKYHD

Nice info any chance and I know it is a big request but


any chance
Those the could not become not relative to the central fanart directory or we add another field to say if it is. That would allow those of us working with other providers to store fanart accessible to phoenix that don't follow the phoenix fanart structure. That is assuming your Fanart class pulls from those fields.

Just a request if not possible I understand but have been considering using the sagedb for some future things and just thinking ahead a bit.

Skye

I made them relative to the fanart folder, because the fanart folder may be different for a client vs a server. ie, ON the server it might be C:\Fanart\ but on a client, it might be F:\Fanart.

Initially I thought about storing the complete path for the image, but then the image would fail to be found on a client, if the client was not using the exact same path as the server.

[PLUCKYHD]

Originally Posted by stuckless

I made them relative to the fanart folder, because the fanart folder may be different for a client vs a server. ie, ON the server it might be C:\Fanart\ but on a client, it might be F:\Fanart.

Initially I thought about storing the complete path for the image, but then the image would fail to be found on a client, if the client was not using the exact same path as the server.

What if you checked a property that i am sure exist then if phoenix fanart is on treat the path as relative if not use the full path? Just trying to think of a way I can tie in with Phoenix but still use other providers storing structure. I know i could just do a imp class for Phoenix fanart but was trying to make it easier for other devs to just rely on this fields.

[stuckless]

Originally Posted by PLUCKYHD

What if you checked a property that i am sure exist then if phoenix fanart is on treat the path as relative if not use the full path? Just trying to think of a way I can tie in with Phoenix but still use other providers storing structure. I know i could just do a imp class for Phoenix fanart but was trying to make it easier for other devs to just rely on this fields.

From the phoenix api, that shouldn't be an issue. If phoenix fanart is not enabled then the the apis won't work anyways. Also, if the value that is returned, is not a valid file, relative to the central fanart folder, then the phoenix apis will go to the central fanart area and load the first available file.

So, you can probably use that field freely, since it will only have value to phoenix in the sense that phoenix will try to use it for the default, if it's relative to the phoenix fanart folder. The other reason for not storing the complete path is that if the fanart folder changes, then you don't need to update this field.

[panteragstk]

What music functions are currently supported? The only thing I'd really want would be background scraping. How hard would that be to enable?

[PLUCKYHD]

Originally Posted by stuckless

From the phoenix api, that shouldn't be an issue. If phoenix fanart is not enabled then the the apis won't work anyways. Also, if the value that is returned, is not a valid file, relative to the central fanart folder, then the phoenix apis will go to the central fanart area and load the first available file.

So, you can probably use that field freely, since it will only have value to phoenix in the sense that phoenix will try to use it for the default, if it's relative to the phoenix fanart folder. The other reason for not storing the complete path is that if the fanart folder changes, then you don't need to update this field.

I was more thinking so phoenix API could support multiple fanart sources

[stuckless]

Originally Posted by PLUCKYHD

I was more thinking so phoenix API could support multiple fanart sources

Phoenix does support more than one source Like almost everything in Phoenix, the fanart is pluggable. If you implement the the IFanartSupport class, and install it (as a plugin), then you can totaly change phoenix fanart. Out of the box, I provide support for the PhoenixFanartSupport. Now, I don't provider compositing where you first look at phoenix suppport, etc,because that would be too slow, but another plugin could implement the IFanartSupport that first checks their fanart location and then if not found, it simply delegates to Phoenix Fanart.

[PLUCKYHD]

Originally Posted by stuckless

Phoenix does support more than one source Like almost everything in Phoenix, the fanart is pluggable. If you implement the the IFanartSupport class, and install it (as a plugin), then you can totaly change phoenix fanart. Out of the box, I provide support for the PhoenixFanartSupport. Now, I don't provider compositing where you first look at phoenix suppport, etc,because that would be too slow, but another plugin could implement the IFanartSupport that first checks their fanart location and then if not found, it simply delegates to Phoenix Fanart.

yeah yeah as above I didn't want to have to rely on a imp class but I can easily go that route. Worth a shot hey

[razrsharpe]

Is there by any chance a tutorial or some documentation on implementing metadata lookups in the STV? I've been thinking about integrating some lookup feature into TVE because sometimes (and i haven't spent the time to figure out why) BMT does not do an automated lookup on newly recorded mediafiles. To be fair I do not know if BMT just does not do the lookup or if the automatic lookup fails or if something else fails. Anyway curious to see if you have any information or can point me to somewhere about implementing a lookup feature in the STV.

[stuckless]

Originally Posted by razrsharpe

Is there by any chance a tutorial or some documentation on implementing metadata lookups in the STV? I've been thinking about integrating some lookup feature into TVE because sometimes (and i haven't spent the time to figure out why) BMT does not do an automated lookup on newly recorded mediafiles. To be fair I do not know if BMT just does not do the lookup or if the automatic lookup fails or if something else fails. Anyway curious to see if you have any information or can point me to somewhere about implementing a lookup feature in the STV.

Before I get your specific question... I would like to know why the lookups fail. I record about 10 shows a week... the only failures that I've seen fall into 2 categories...
1. Can't find the series
2. TVDB has not been updated (ie no specific episode information)

Too be honest, it's rare that I don't get a hit. The phoenix.log and phoenix-metadata.log files should contain information about the failure.

OK, so here are the metadata scanning apis... (taken from http://code.google.com/p/sagephoenix/wiki/APISummary)

phoenix_metadatascan_StartMetadataScan(Object source, Object options): Object
phoenix_metadatascan_IsMetadataScanRunning(Object progress): boolean
phoenix_metadatascan_GetMetadataScanComplete(Object tracker): float
phoenix_metadatascan_CancelMetadataScan(Object tracker): boolean
phoenix_metadatascan_GetMetadataSearchResults(Object mediaFile): IMetadataSearchResult[]
phoenix_metadatascan_GetMetadataSearchResultTitle(IMetadataSearchResult result): String
phoenix_metadatascan_GetMetadataSearchResultScore(IMetadataSearchResult result): float
phoenix_metadatascan_GetMetadataSearchResultYear(IMetadataSearchResult result): String
phoenix_metadatascan_GetMetadataSearchResultProviderId(IMetadataSearchResult result): String
phoenix_metadatascan_GetMetadataSearchResults(Object mediaFile, String name, String type): IMetadataSearchResult[]
phoenix_metadatascan_UpdateMediaFileMetadata(Object mediaFile, IMetadataSearchResult result): void
phoenix_metadatascan_UpdateMediaFileMetadata(Object mediaFile, IMetadataSearchResult result, boolean wait, Object options): void
phoenix_metadatascan_GetMetadataProgressFailedCount(Object progress): int
phoenix_metadatascan_GetMetadataProgressSuccessCount(Object progress): int
phoenix_metadatascan_GetMetadataProgressSkippedCount(Object progress): int
phoenix_metadatascan_GetMetadataFailedItems(Object progress): Object[]
phoenix_metadatascan_GetMetadataSuccessItems(Object progress): Object[]
phoenix_metadatascan_GetMetadataSkippedItems(Object progress): Object[]
phoenix_metadatascan_GetMetadataScanTrackers(): Object[]
phoenix_metadatascan_CreateHints(): Map<String, String>

In a nutshell... you pass a File or sage MediaFile to

phoenix_metadatascan_GetMetadataSearchResults(Object mediaFile): IMetadataSearchResult[]

Then you can display a list of result details using, phoenix_metadatascan_GetMetadataSearchResultTitle/Year/Score, etc.

ANd when the user selects a result you call

phoenix_metadatascan_UpdateMediaFileMetadata(Object mediaFile, IMetadataSearchResult result):

Passing in the mediafile and the result. This will run in the current thread, which can take awhile, so you may want to fork in the UI an call this api.

You can PM me with other details.

EDIT: Since I don't return a status (which I should) you'll never know if it fails. For movies it should never fail, but for TV, it can since the actual episode lookups don't happen until UpdateMediaFileMetadata is called

[razrsharpe]

heres a snippet from phoenix-metadata.log from thursday nite... 2 shows that ended at 10pm and failed to look up. There was also a show that ended at 10:01 and that lookup went fine.

2011-02-03 22:00:00,447 - AUTO; RECORDING; \\svr1\TV\Nikita-CoupdeGrace-4140372-0.mpg; Nikita; 4218784
2011-02-03 22:00:11,197 - ERROR; \\svr1\TV\Nikita-CoupdeGrace-4140372-0.mpg; Nikita; 4218784; Search Failed for SearchQuery [type=TV, fields={CLEAN_TITLE: Nikita,EPISODE_TITLE: Coup de Grace,RAW_TITLE: Nikita,QUERY: Nikita,YEAR: 0,EPISODE_DATE: 2011-02-03,}, hints=Hints [hints={update_fanart: true,update_metadata: true,scan_missing_metadata: true,scan_subfolders: true,known_recording: true,import_tv_as_recording: false,auto: true,}]]
2011-02-03 22:00:11,197 - AUTO; RECORDING; \\svr1\TV\Bones-TheSinintheSisterhood-4140114-0.mpg; Bones; 4218783
2011-02-03 22:00:21,869 - ERROR; \\svr1\TV\Bones-TheSinintheSisterhood-4140114-0.mpg; Bones; 4218783; Search Failed for SearchQuery [type=TV, fields={CLEAN_TITLE: Bones,EPISODE_TITLE: The Sin in the Sisterhood,RAW_TITLE: Bones,QUERY: Bones,YEAR: 0,EPISODE_DATE: 2011-02-03,}, hints=Hints [hints={update_fanart: true,update_metadata: true,scan_missing_metadata: true,scan_subfolders: true,known_recording: true,import_tv_as_recording: false,auto: true,}]]

maybe tvdb's servers were overloaded at the top of the hour? I did check on tvdb and info does exist for the episodes that aired...

Heres the snippet from the phoenix.log

2011-02-03 22:00:00,447 [Timer-19] INFO  sagex.phoenix.metadata.MetadataManager - search(): tvdb; SearchQuery [type=TV, fields={CLEAN_TITLE: Nikita,EPISODE_TITLE: Coup de Grace,RAW_TITLE: Nikita,YEAR: 0,EPISODE_DATE: 2011-02-03,}, hints=Hints [hints={update_fanart: true,update_metadata: true,scan_missing_metadata: true,scan_subfolders: true,known_recording: true,import_tv_as_recording: false,auto: true,}]]
2011-02-03 22:00:00,463 [Timer-19] INFO  sagex.phoenix.metadata.MetadataManager - Searching: Nikita using TVDBMetadataProvider[MetadataProviderInfo [id=tvdb, fanartId=null, mediaTypes=[TV], name=thetvdb.com]]
2011-02-03 22:00:00,463 [Timer-19] INFO  sagex.phoenix.metadata.provider.tvdb.TVDBSearchParser - TVDB Search: http://www.thetvdb.com/api/GetSeries.php?seriesname=Nikita
2011-02-03 22:00:11,197 [Timer-19] ERROR sagex.phoenix.metadata.provider.tvdb.TVDBSearchParser - Failed to parse/search using url: CachedUrl: http://www.thetvdb.com/api/GetSeries.php?seriesname=Nikita; UrlId: d90033d99b393d3f4ae0b088e33c0abd
java.net.SocketTimeoutException: Read timed out
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.read(Unknown Source)
    at java.io.BufferedInputStream.fill(Unknown Source)
    at java.io.BufferedInputStream.read1(Unknown Source)
    at java.io.BufferedInputStream.read(Unknown Source)
    at sun.net.www.http.HttpClient.parseHTTPHeader(Unknown Source)
    at sun.net.www.http.HttpClient.parseHTTP(Unknown Source)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
    at sagex.phoenix.util.url.CachedUrl.cache(CachedUrl.java:156)
    at sagex.phoenix.util.url.CachedUrl.getUrl(CachedUrl.java:141)
    at sagex.phoenix.util.url.CachedUrl.getInputStream(CachedUrl.java:199)
    at sagex.phoenix.metadata.provider.tvdb.TVDBSearchParser.getResults(TVDBSearchParser.java:78)
    at sagex.phoenix.metadata.provider.tvdb.TVDBMetadataProvider.search(TVDBMetadataProvider.java:47)
    at sagex.phoenix.metadata.MetadataManager.search(MetadataManager.java:357)
    at sagex.phoenix.metadata.MetadataManager.automaticUpdate(MetadataManager.java:572)
    at sagex.phoenix.metadata.MetadataManager.automaticUpdate(MetadataManager.java:542)
    at sagex.phoenix.plugin.PhoenixPlugin.updateMetadata(PhoenixPlugin.java:147)
    at sagex.phoenix.plugin.PhoenixPlugin.recordingCompleted(PhoenixPlugin.java:134)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at sagex.plugin.AbstractPlugin$1.run(AbstractPlugin.java:254)
    at java.util.TimerThread.mainLoop(Unknown Source)
    at java.util.TimerThread.run(Unknown Source)
2011-02-03 22:00:11,197 [Timer-19] WARN  phoenix.log - ERROR; \\svr1\TV\Nikita-CoupdeGrace-4140372-0.mpg; Nikita; 4218784; Search Failed for SearchQuery [type=TV, fields={CLEAN_TITLE: Nikita,EPISODE_TITLE: Coup de Grace,RAW_TITLE: Nikita,QUERY: Nikita,YEAR: 0,EPISODE_DATE: 2011-02-03,}, hints=Hints [hints={update_fanart: true,update_metadata: true,scan_missing_metadata: true,scan_subfolders: true,known_recording: true,import_tv_as_recording: false,auto: true,}]]
MetadataException [query=SearchQuery [type=TV, fields={CLEAN_TITLE: Nikita,EPISODE_TITLE: Coup de Grace,RAW_TITLE: Nikita,QUERY: Nikita,YEAR: 0,EPISODE_DATE: 2011-02-03,}, hints=Hints [hints={update_fanart: true,update_metadata: true,scan_missing_metadata: true,scan_subfolders: true,known_recording: true,import_tv_as_recording: false,auto: true,}]], ]
    at sagex.phoenix.metadata.MetadataManager.search(MetadataManager.java:386)
    at sagex.phoenix.metadata.MetadataManager.automaticUpdate(MetadataManager.java:572)
    at sagex.phoenix.metadata.MetadataManager.automaticUpdate(MetadataManager.java:542)
    at sagex.phoenix.plugin.PhoenixPlugin.updateMetadata(PhoenixPlugin.java:147)
    at sagex.phoenix.plugin.PhoenixPlugin.recordingCompleted(PhoenixPlugin.java:134)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at sagex.plugin.AbstractPlugin$1.run(AbstractPlugin.java:254)
    at java.util.TimerThread.mainLoop(Unknown Source)
    at java.util.TimerThread.run(Unknown Source)
2011-02-03 22:00:11,197 [Timer-19] INFO  sagex.phoenix.metadata.MetadataManager - search(): tvdb; SearchQuery [type=TV, fields={CLEAN_TITLE: Bones,EPISODE_TITLE: The Sin in the Sisterhood,RAW_TITLE: Bones,YEAR: 0,EPISODE_DATE: 2011-02-03,}, hints=Hints [hints={update_fanart: true,update_metadata: true,scan_missing_metadata: true,scan_subfolders: true,known_recording: true,import_tv_as_recording: false,auto: true,}]]
2011-02-03 22:00:11,197 [Timer-19] INFO  sagex.phoenix.metadata.MetadataManager - Searching: Bones using TVDBMetadataProvider[MetadataProviderInfo [id=tvdb, fanartId=null, mediaTypes=[TV], name=thetvdb.com]]
2011-02-03 22:00:11,197 [Timer-19] INFO  sagex.phoenix.metadata.provider.tvdb.TVDBSearchParser - TVDB Search: http://www.thetvdb.com/api/GetSeries.php?seriesname=Bones
2011-02-03 22:00:21,869 [Timer-19] ERROR sagex.phoenix.metadata.provider.tvdb.TVDBSearchParser - Failed to parse/search using url: CachedUrl: http://www.thetvdb.com/api/GetSeries.php?seriesname=Bones; UrlId: d1439c9d7326e1c502e59681561c929c
java.net.SocketTimeoutException: Read timed out
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.read(Unknown Source)
    at java.io.BufferedInputStream.fill(Unknown Source)
    at java.io.BufferedInputStream.read1(Unknown Source)
    at java.io.BufferedInputStream.read(Unknown Source)
    at sun.net.www.http.HttpClient.parseHTTPHeader(Unknown Source)
    at sun.net.www.http.HttpClient.parseHTTP(Unknown Source)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
    at sagex.phoenix.util.url.CachedUrl.cache(CachedUrl.java:156)
    at sagex.phoenix.util.url.CachedUrl.getUrl(CachedUrl.java:141)
    at sagex.phoenix.util.url.CachedUrl.getInputStream(CachedUrl.java:199)
    at sagex.phoenix.metadata.provider.tvdb.TVDBSearchParser.getResults(TVDBSearchParser.java:78)
    at sagex.phoenix.metadata.provider.tvdb.TVDBMetadataProvider.search(TVDBMetadataProvider.java:47)
    at sagex.phoenix.metadata.MetadataManager.search(MetadataManager.java:357)
    at sagex.phoenix.metadata.MetadataManager.automaticUpdate(MetadataManager.java:572)
    at sagex.phoenix.metadata.MetadataManager.automaticUpdate(MetadataManager.java:542)
    at sagex.phoenix.plugin.PhoenixPlugin.updateMetadata(PhoenixPlugin.java:147)
    at sagex.phoenix.plugin.PhoenixPlugin.recordingCompleted(PhoenixPlugin.java:134)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at sagex.plugin.AbstractPlugin$1.run(AbstractPlugin.java:254)
    at java.util.TimerThread.mainLoop(Unknown Source)
    at java.util.TimerThread.run(Unknown Source)
2011-02-03 22:00:21,869 [Timer-19] WARN  phoenix.log - ERROR; \\svr1\TV\Bones-TheSinintheSisterhood-4140114-0.mpg; Bones; 4218783; Search Failed for SearchQuery [type=TV, fields={CLEAN_TITLE: Bones,EPISODE_TITLE: The Sin in the Sisterhood,RAW_TITLE: Bones,QUERY: Bones,YEAR: 0,EPISODE_DATE: 2011-02-03,}, hints=Hints [hints={update_fanart: true,update_metadata: true,scan_missing_metadata: true,scan_subfolders: true,known_recording: true,import_tv_as_recording: false,auto: true,}]]
MetadataException [query=SearchQuery [type=TV, fields={CLEAN_TITLE: Bones,EPISODE_TITLE: The Sin in the Sisterhood,RAW_TITLE: Bones,QUERY: Bones,YEAR: 0,EPISODE_DATE: 2011-02-03,}, hints=Hints [hints={update_fanart: true,update_metadata: true,scan_missing_metadata: true,scan_subfolders: true,known_recording: true,import_tv_as_recording: false,auto: true,}]], ]
    at sagex.phoenix.metadata.MetadataManager.search(MetadataManager.java:386)
    at sagex.phoenix.metadata.MetadataManager.automaticUpdate(MetadataManager.java:572)
    at sagex.phoenix.metadata.MetadataManager.automaticUpdate(MetadataManager.java:542)
    at sagex.phoenix.plugin.PhoenixPlugin.updateMetadata(PhoenixPlugin.java:147)
    at sagex.phoenix.plugin.PhoenixPlugin.recordingCompleted(PhoenixPlugin.java:134)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at sagex.plugin.AbstractPlugin$1.run(AbstractPlugin.java:254)
    at java.util.TimerThread.mainLoop(Unknown Source)
    at java.util.TimerThread.run(Unknown Source)

my uneducated look at the logs it looks like there was a timeout error doing the search... If that's the case maybe it would be possible to requeue the lookup for a later time???

Also attached are the full logs.... In the past week I've had 3 failures out of ~34.... I think usually its 1 or 2 per week that fail...


Thanks for the info on doing the lookup from the stv... I'll let you know when i have questions

EDIT: First Question When doing the UpdateMediaFileMetadata does it honor the "preserve original metadata" property? I ask because sometimes the description for shows from the epg is empty... I'd like to have an easy way to update it from the ui (reason 2 for the lookup from the stv)... Or even better... have bmt detect that the description is empty and update it during the automatic lookup if and only if the epg description is empty.

[stuckless]

Yeah, the update does honor the preserve metadata (at least it should). I've added a request for the update empty descriptions when preserve metadata is enabled, and a request for the re-queue, which is come up a number of times