I want to build a Silverlight live feed viewer for an IP camera with a proprietary RTP server, i.e. no IIS, no SmoothStreaming extension. Is SmoothStreamingClient (or microsoft media platform) is the best place to start?
You definitely don't want the SmoothStreamingClient, as that assumes that you're using a SmoothStreaming media source. However, what you can do instead is use a MediaElement and implement your own MediaStreamSource. This requires that you know how to parse the data being spewed by your IP camera and turn it into valid video samples, which is non-trivial, but it's the only supplied mechanism for displaying video data for which there isn't already a built-in streaming source.
However, if the video format that your IP camera sends is already supported by Silverlight, then all you need to do is create a Stream that reads the camera data and pass that as the media source to a MediaElement.