This is the revised version of commit 448fc5c.
Commit 448fc5c has a critical regression in scroll tracking that can cause the
scroll content to bounce after moving a panel, for example, pulling down it from
full to half state.
By re-investigating #524, I found that this problem only occurred with the
`fitToBounds` content mode and a small scroll view content.
Therefore I fixed it in the more specific way.