关于speak的搭配问题

I konw her by sight, but not to speak to. 译：我只不过认得她，但没跟她说过话。but后面是什么结构，或者完整形式是什么，有点不懂

I know her by sight, but not to speak to. 这是口语中的省略。
= I know her by sight, but I don't know her enough to speak to.