The castle of Velo had been granted from the eleventh century in fief to the counts of Velo by the Bishops of Vicenza. It is known from a public act reported by Pagliarino that "in 1230 Olderico and Judge Castellano divided the castle of Velo but not the tower for the defence against the wiles of their enemies". The building must have been very robust, if it is true that the very Ezzelino the Tyrant did not dare to attack it.