Library music is generally music not written for a specific film or project.
As such, an original soundtrack is not library music. A soundtrack would generally not be considered library music, unless in the odd case where:
1) The soundtrack in part or in its entirety is sourced from a library.
2) Cues specifically composed for this somehow ended up for reusage in a library later on. (rarer)
Note that in both cases above, this is not dependent on or defined by inclusion in a film, but rather that the music belongs to a library.