No, not necessarily, for the second question (with $ g $).
For example, let $ f ( x ) $ be $ x ^ { 1 / 2 } \sin ( 1 / x ) $; then $ f $ is continuous on $ ( 0 , 1 ] $ with $ \lim \limits _ { x \to 0 } f ( x ) = 0 $. And let $ g ( x ) $ be $ x ^ { - 3 / 2 } \sin ( 1 / x ) $; then $ g $ is continuous on $ ( 0 , 1 ] $ and improperly integrable on $ [ 0 , 1 ] $, with $ \int _ { x = 0 } ^ 1 g ( x ) \, \mathrm d x \approx 0 .6 3 $ (see WolframAlpha for more digits). Then $ f ( x ) g ( x ) = x ^ { - 1 } \sin ^ 2 ( 1 / x ) $; now $ f g $ is not improperly integrable on $ [ 0 , 1 ] $; instead, $ \int _ { x = 0 } ^ 1 f ( x ) g ( x ) \, \mathrm d x = \infty $.
Note that $ g $ is not Lebesgue integrable. If you assume that it is, then the answer is Yes. This includes the first question as a special case, with $ g ( x ) = 1 $. It also includes your specific example with $ g ( x ) = x ^ { - 3 / 2 } ( x + x ^ a ) $, since that's always positive.
Commentary: The key to this is to make $ g $ an improperly Riemann-integrable function that's not properly Lebesgue-integrable (so that $ g $ must alternate between positive and negative values), then multiply this by a continuous (or continuously extendible) function $ f $ to get a positive function (so that $ f $ must also alternate sign). The standard example for $ g $ is to use $ g ( x ) = x ^ { - 1 } \sin ( 1 / x ) $, but anything of the form $ g ( x ) = x ^ \alpha \sin ( 1 / x ) $ will work as long as $ - 2 < \alpha \leq - 1 $ (where $ \alpha > - 2 $ ensures that it's integrable and $ \alpha \leq - 1 $ ensures that it's not absolutely integrable). The standard example for $ f $ (given that its sign must match that of $ g $) is to use $ f ( x ) = x \sin ( 1 / x ) $, but anything of the form $ f ( x ) = x ^ \beta \sin ( 1 / x ) $ will work as long as $ \beta > 0 $. Multiplying these gives $ x ^ { \alpha + \beta } \sin ^ 2 ( 1 / x ) $, which will diverge if $ \alpha + \beta \leq - 1 $. So using the standard examples, with $ \alpha = - 1 $ and $ \beta = 1 $, will fail; but there's some room for fudging, and using $ \alpha = - 3 / 2 $ and $ \beta = 1 / 2 $ works.