A Saudi law professor is detained for writing an article about a possible power struggle over the royal succession, a rights group says.