All of the previous answers are good, but still theoretical. Here is my perspective as a Software Quality professional and what I expect developers to test.

1 Test that your feature / bug fix / etc meets the defined acceptance criteria
2 Test any and all logic paths created/updated during development
3 If building UI features / components, do a sanity test across different browsers (i.e. IE)

With the three points above and considering your final question. To me it means you'd test the combination of components produced by React (manually or automated). If you are creating dynamic components based on specific data or behavior, then that is a logical path that should be tested. If the result of the logic path is a React component, then you can write automated tests that validate the component was actually generated.