A minister has insisted that devolution arrangements are not being decided centrally, after opposition peers accused the government of “taking power away” from communities.