Part of the walls are all that remain of the old castle complex which probably dates back to Moorish times.