Playback can only start at the closest sequence header.
On a stock DVD, this isn’t an issue, because the sequence headers are half a second apart, so you can start “anywhere” within the video stream within a half-second accuracy.
Other encodes have longer sequences between headers. Some PVR files have only one header for the entire file – they can’t be FF at all – they can only be played from the beginning.
Files encoded with Handbrake, for example, have the headers about 5 seconds apart, on average. So even if the chapter information contains the exact frame that chapter is supposed to start at, when you actually select that chapter, playback starts at the nearest header – either a couple of seconds before, or a couple of seconds after the actual time the chapter is coded for. There’s no way to force playback to begin at the exact frame specified, unless it happens to also be the start of a sequence.
Some other encoders can create even longer sequences. Many use a GOP length of 300 frames, and then group several GOPs into one sequence… playback can be 10 minutes off from where you expect it to be. This leads to odd quirks like pressing fast-forward for a few seconds, and when you press play again, the playback begins 5 minutes before you pressed fast-forward.