No, an AI can't create it's rules as it goes. Or, at least, those rules aren't likely to be based on anything functional without a good (large) starting set.
Unlike what you see in fiction, current AIs aren't actually capable of though. They just use their data set to calculate the most...