Formation of terrestrial planets in eccentric and inclined giant planet systems