`Error class: unknown class` in KMP Multi-module project about dokkatoo

[chrisbanes]

Ha, yeah I just tried the same thing and it works as expected. Thanks for the quick response!

[aSemy]

Does the Dokka plugin output a JSON file like Dokkatoo? We have a working Dokka setup so should be pretty easy to diff if it exists

Unfortunately not Kotlin/dokka#2873, but it is possible to add a breakpoint in Dokka and debug into the code - but it's not very convenient

[aSemy]

Thanks for the report. From looking at the dokka_parameters.json files it looks like the classpath does not include local dependencies.

For example, the generated function public fun Column(...) has arguments of type app.cash.redwood.layout.api.Constraint, which comes from subproject :redwood-layout-api, but the source set in redwood-layout-compose/build/dokka-config/html/dokka_parameters.json doesn't contain any reference to :redwood-layout-api files.

It might be caused by this line, which filters out non-project dependencies from the classpath, and I can't remember why I put it in... It can probably be removed without issue.

dokkatoo/modules/dokkatoo-plugin/src/main/kotlin/adapters/DokkatooKotlinAdapter.kt

Line 229 in 8f735f0

componentFilter(!LocalProjectOnlyFilter)

[aSemy]

I've done a quick hacky test and it looks like removing

dokkatoo/modules/dokkatoo-plugin/src/main/kotlin/adapters/DokkatooKotlinAdapter.kt

Line 229 in 8f735f0

componentFilter(!LocalProjectOnlyFilter)

fixes the problem.

I'll push a fix soon and you can try the snapshot @chrisbanes

[chrisbanes]

Just tried out more of the built docs and while the references are now fixed, but it seems that some of the links don't work (the linked docs don't exist). A key example is in docs/build/dokkatoo/html/redwood-treehouse-composeui/app.cash.redwood.treehouse.composeui/-treehouse-content.html. None of the docs for the classes exist.

[aSemy]

Sorry, I don't understand what you mean. Could you share a screenshot of the docs, and a link to the source code of the missing classes?

When I look in Redwood's current docs, I can see fun TreehouseContent(...)

And the same function in the Dokkatoo generated docs

[chrisbanes]

Try and click on those links, most will 404 (CodeListener for example)

[aSemy]

Gotcha, I see what you mean.

Clicking on CodeListener in Dokkatoo links to

redwood/docs/build/dokkatoo/html/redwood-treehouse/app.cash.redwood.treehouse/-code-listener/index.html

While it should link to

redwood/docs/build/dokkatoo/html/redwood-treehouse-host/app.cash.redwood.treehouse/-code-listener/index.html

I'll have a look around but I'm not sure what the cause might be. Dokkatoo is basically a fancy wrapper around the Dokka config, so presumably Dokkatoo is generating slightly incorrect config. The trick is to figure out why Dokkatoo is trying to link to redwood-treehouse instead of redwood-treehouse-host

[chrisbanes]

Does the Dokka plugin output a JSON file like Dokkatoo? We have a working Dokka setup so should be pretty easy to diff if it exists

[aSemy]

Interestingly if I remove the subproject logic and add explicit dependencies then the links seems to work

// docs/build.gradle

//rootProject.subprojects.each {
//  if (it.name == name) return
//  evaluationDependsOn(it.path)
//
//  // Each submodule needs to have the dokkatoo plugin applied
//  if (!it.plugins.hasPlugin('dev.adamko.dokkatoo-html')) return
//
//  dependencies {
//    dokkatoo(project(it.path))
//  }
//}

dependencies {
  dokkatoo projects.redwoodTreehouseComposeui
  dokkatoo projects.redwoodTreehouseHost

  // Need to manually depend on this due to https://github.com/adamko-dev/dokkatoo/issues/14
  dokkatooPluginHtml("org.jetbrains.dokka:all-modules-page-plugin:1.8.10")
  //dokkatooPluginHtml("org.jetbrains.dokka:templating-plugin:1.8.10") // shouldn't be necessary
}

[chrisbanes]

Ah interesting! I'll have a play around and see if I can get it to work somehow.

[aSemy]

However, if I add all dependencies manually, then clicking CodeListener 404's again!

dependencies {
//  dokkatoo projects.redwoodCli
  dokkatoo projects.redwoodCompose
  dokkatoo projects.redwoodComposeTesting
  dokkatoo projects.redwoodFlexbox
  dokkatoo projects.redwoodGradlePlugin
  dokkatoo projects.redwoodLayoutApi
  //dokkatoo projects.redwoodLayoutApiDensity
  dokkatoo projects.redwoodLayoutCompose
  dokkatoo projects.redwoodLayoutComposeui
  dokkatoo projects.redwoodLayoutDom
  dokkatoo projects.redwoodLayoutLayoutmodifiers
  dokkatoo projects.redwoodLayoutSchema
//  dokkatoo projects.redwoodLayoutSharedTest
  dokkatoo projects.redwoodLayoutTesting
  dokkatoo projects.redwoodLayoutUiview
  dokkatoo projects.redwoodLayoutView
  dokkatoo projects.redwoodLayoutWidget
  dokkatoo projects.redwoodProtocol
  dokkatoo projects.redwoodProtocolCompose
  dokkatoo projects.redwoodProtocolWidget
  dokkatoo projects.redwoodRuntime
  dokkatoo projects.redwoodSchema
//  dokkatoo projects.redwoodToolingCodegen
//  dokkatoo projects.redwoodToolingLint
//  dokkatoo projects.redwoodToolingSchema
  dokkatoo projects.redwoodTreehouse
  dokkatoo projects.redwoodTreehouseComposeui
  dokkatoo projects.redwoodTreehouseComposeuiInsets
  dokkatoo projects.redwoodTreehouseConfiguration
  dokkatoo projects.redwoodTreehouseGuest
  dokkatoo projects.redwoodTreehouseGuestCompose
  dokkatoo projects.redwoodTreehouseHost
  dokkatoo projects.redwoodTreehouseLazylayoutApi
  dokkatoo projects.redwoodTreehouseLazylayoutCompose
  dokkatoo projects.redwoodTreehouseLazylayoutComposeui
  dokkatoo projects.redwoodTreehouseLazylayoutPaging
  dokkatoo projects.redwoodTreehouseLazylayoutSchema
  dokkatoo projects.redwoodTreehouseLazylayoutTesting
  dokkatoo projects.redwoodTreehouseLazylayoutUiview
  dokkatoo projects.redwoodTreehouseLazylayoutView
  dokkatoo projects.redwoodTreehouseLazylayoutWidget
  dokkatoo projects.redwoodWidget
  dokkatoo projects.redwoodWidgetCompose

  // Need to manually depend on this due to https://github.com/adamko-dev/dokkatoo/issues/14
  dokkatooPluginHtml("org.jetbrains.dokka:all-modules-page-plugin:1.8.10")
//  dokkatooPluginHtml("org.jetbrains.dokka:templating-plugin:1.8.10")
}

[chrisbanes]

Unfortunately not Kotlin/dokka#2873, but it is possible to add a breakpoint in Dokka and debug into the code - but it's not very convenient

Any tips on how to debug/breakpoint this? I've tried a few things and can't get it to break. I guess diff'ing the configuration is the only way we're going to track this down.

[aSemy]

Hmm no sorry, I thought there was a thread in the Dokka issue board explaining, but I can't find it any more. You could try asking on the #dokka Slack?

Dokkatoo and Dokka should generate identical content, so while even if it's not possible to check the JSON configuration, you can do a diff on build/dokka/htmlMultiModule vs docs/build/dokkatoo, or on a module basis (e.g. redwood-treehouse-lazylayout-composeui/build/dokka-module/html vs redwood-treehouse-lazylayout-composeui/build/dokka/htmlPartial).

I've done some digging and unfortunately I haven't found an answer.

One thing that I thought was interesting is that the package-list file that Dokka-classic generates (build/dokka/htmlMultiModule/package-list) is much larger than the file Dokkatoo generates (docs/build/dokkatoo/html/package-list).

EDIT: never mind, I had disabled most of the subprojects, the package-lists are actually the same

Dokka's package-list

$dokka.format:html-v1
$dokka.linkExtension:html

module:redwood-compose
app.cash.redwood.compose
module:redwood-compose-testing
app.cash.redwood.compose.testing
module:redwood-flexbox
app.cash.redwood.flexbox
module:redwood-layout-api
app.cash.redwood.layout.api
module:redwood-layout-compose
app.cash.redwood.layout.compose
module:redwood-layout-composeui
app.cash.redwood.layout.composeui
module:redwood-layout-dom
app.cash.redwood.layout.dom
module:redwood-layout-layoutmodifiers
app.cash.redwood.layout
module:redwood-layout-schema
app.cash.redwood.layout
module:redwood-layout-testing
app.cash.redwood.layout.widget
module:redwood-layout-uiview
app.cash.redwood.layout.uiview
module:redwood-layout-view
app.cash.redwood.layout.view
module:redwood-layout-widget
app.cash.redwood.layout.widget
module:redwood-protocol
app.cash.redwood.protocol
module:redwood-protocol-compose
app.cash.redwood.protocol.compose
module:redwood-protocol-widget
app.cash.redwood.protocol.widget
module:redwood-runtime
app.cash.redwood
module:redwood-treehouse
app.cash.redwood.treehouse
module:redwood-treehouse-composeui
app.cash.redwood.treehouse.composeui
module:redwood-treehouse-composeui-insets
app.cash.redwood.treehouse.composeui
module:redwood-treehouse-configuration
app.cash.redwood.treehouse
module:redwood-treehouse-guest
app.cash.redwood.treehouse
module:redwood-treehouse-guest-compose
app.cash.redwood.treehouse
module:redwood-treehouse-host
app.cash.redwood.treehouse
module:redwood-treehouse-lazylayout-api
app.cash.redwood.treehouse.lazylayout.api
module:redwood-treehouse-lazylayout-compose
app.cash.redwood.treehouse.lazylayout.compose
module:redwood-treehouse-lazylayout-composeui
app.cash.redwood.treehouse.lazylayout.composeui
module:redwood-treehouse-lazylayout-paging
app.cash.redwood.treehouse.lazylayout.paging
module:redwood-treehouse-lazylayout-schema
app.cash.redwood.treehouse.lazylayout
module:redwood-treehouse-lazylayout-testing
app.cash.redwood.treehouse.lazylayout.widget
module:redwood-treehouse-lazylayout-uiview
app.cash.redwood.treehouse.lazylayout.uiview
module:redwood-treehouse-lazylayout-view
app.cash.redwood.treehouse.lazylayout.view
module:redwood-treehouse-lazylayout-widget
app.cash.redwood.treehouse.lazylayout.widget
module:redwood-widget
app.cash.redwood.widget
module:redwood-widget-compose
app.cash.redwood.widget.compose

Dokkatoo's package-list

$dokka.format:html-v1
$dokka.linkExtension:html

module:redwood-compose
app.cash.redwood.compose
module:redwood-treehouse-composeui
app.cash.redwood.treehouse.composeui
module:redwood-treehouse-composeui-insets
app.cash.redwood.treehouse.composeui
module:redwood-treehouse-configuration
app.cash.redwood.treehouse
module:redwood-treehouse-guest
app.cash.redwood.treehouse
module:redwood-treehouse-guest-compose
app.cash.redwood.treehouse
module:redwood-treehouse-host
app.cash.redwood.treehouse

[aSemy]

I've done a lot of digging over the weekend, and I came up with something to show the Dokka configuration.

1. Checkout Dokka

2. Edit DokkaGenerator to log the config, and save it to file

   // core/src/main/kotlin/DokkaGenerator.kt
   
   class DokkaGenerator(
       private val configuration: DokkaConfiguration,
       private val logger: DokkaLogger
   ) {
       init {
           val configJson = serializeAsPrettyJson(configuration)
           println("loaded DokkaGenerator with configuration: $configJson")
           configuration.cacheRoot
               ?.resolve("dokka-configuration-${currentTimeMillis()}.json")
               ?.apply { parentFile.mkdirs() }
               ?.apply { createNewFile() }
               ?.writeText(configJson)
       }

3. Publish Dokka to Maven Local

4. Add mavenLocal() to Redwood's build config (I limit the group to avoid problems)

   repositories {
     google()
     mavenCentral()
     gradlePluginPortal()
     mavenLocal {
       mavenContent {
         includeGroup("org.jetbrains.dokka")
       }
     }
   }

5. Configure the cache dir for Dokka tasks

   // build.gradle.kts
   tasks.withType<org.jetbrains.dokka.gradle.AbstractDokkaTask> {
     cacheRoot.set(temporaryDir)
   }

6. Configure Dokkatoo to set both the cache dir and the Dokka version

   // build.gradle.kts
   dokkatoo {
     dokkatooCacheDirectory.set(layout.buildDirectory.dir("tmp/.dokkatoo/cache-dir"))
     
     versions {
       jetbrainsDokka.set("1.8.20-SNAPSHOT")
     }
   
     // I also added this so that the Dokka config would be more similar - but it's not necessary, doesn't affect the generated publication, and should be removed!
     sourceSetScopeDefault.set(project.path + ":dokkaHtmlPartial")
   }

Now run Dokka and Dokkatoo!

• ./gradlew :docs:dokkatooGenerate
• ./gradlew dokkaHtmlMultiModule

I've compared the Dokka config of :redwood-widget (redwood-widget/build/tmp/dokkaHtmlPartial/dokka-configuration-16....json vs redwood-widget/build/tmp/.dokkatoo/cache-dir/dokka-configuration-16....json) and found some inconsistencies, mostly with naming, but the ones that I think are problematic are related to the source set with displayName osArm64

      // ID of the problematic source set
      "displayName": "iosArm64",
      "sourceSetID": {
        "scopeId": ":redwood-widget:dokkaHtmlPartial",
        "sourceSetName": "iosArm64Main"
      },

In Dokka the classpath contains a lot of Kotlin Native files, but these are completely absent from the Dokkatoo config.

  // why are these classpath files missing in Dokkatoo?
  "classpath": [
    ...
    "/Users/me/.local/share/konan/kotlin-native-prebuilt-macos-x86_64-1.8.20/klib/platform/ios_arm64/org.jetbrains.kotlin.native.platform.CoreFoundation",
    "/Users/me/.local/share/konan/kotlin-native-prebuilt-macos-x86_64-1.8.20/klib/platform/ios_arm64/org.jetbrains.kotlin.native.platform.Network",
    "/Users/me/.local/share/konan/kotlin-native-prebuilt-macos-x86_64-1.8.20/klib/platform/ios_arm64/org.jetbrains.kotlin.native.platform.QuartzCore",
    "/Users/me/.local/share/konan/kotlin-native-prebuilt-macos-x86_64-1.8.20/klib/platform/ios_arm64/org.jetbrains.kotlin.native.platform.MLCompute",
    "/Users/me/.local/share/konan/kotlin-native-prebuilt-macos-x86_64-1.8.20/klib/platform/ios_arm64/org.jetbrains.kotlin.native.platform.SafetyKit",
    "/Users/me/.local/share/konan/kotlin-native-prebuilt-macos-x86_64-1.8.20/klib/platform/ios_arm64/org.jetbrains.kotlin.native.platform.Security",
    "/Users/me/.local/share/konan/kotlin-native-prebuilt-macos-x86_64-1.8.20/klib/platform/ios_arm64/org.jetbrains.kotlin.native.platform.CloudKit",
    ...
  ]

In the Dokka config the dependentSourceSets includes iosMain

      "dependentSourceSets": [
        {
          "scopeId": ":redwood-widget:dokkaHtmlPartial",
          "sourceSetName": "iosMain"
        },
        {
          "scopeId": ":redwood-widget:dokkaHtmlPartial",
          "sourceSetName": "commonMain"
        }
      ],

While in Dokkatoo iosMain is missing...

      // why is iosMain missing?
      "dependentSourceSets": [
        {
          "scopeId": ":redwood-widget:dokkaHtmlPartial",
          "sourceSetName": "commonMain"
        }
      ],

I'm not sure what to do about this! The Dokka config that extracts the source set/dependencies information is not compatible with Gradle configuration cache, which Dokkatoo needs to adhere to.

[chrisbanes]

Interesting! This might be the same issue which is blocking upgrade to Gradle 8.x: cashapp/redwood#823

I haven't looked too deeply, but it seems that the KMP plugin does some interesting things wrt Gradle tasks, and Dokka has explicit support for KMP projects.

[aSemy]

Hmm yes, they are probably related. My guess is the error is caused by Dokka fetching the dependencies using the Kotlin Compile task

https://github.com/Kotlin/dokka/blob/dd2a38b12658e7343a1357dc5cd3850fa090a4ba/runners/gradle-plugin/src/main/kotlin/org/jetbrains/dokka/gradle/kotlin/kotlinClasspathUtils.kt#L52-L60

But Dokkatoo won't do this, because it's not compatible with Gradle config-cache.

[aSemy]

tl;dr:

• missing dependant source set: I found a Dokkatoo bug that I'll fix.
• files missing from classpath: fixable, but triggers a bug in KGP (a workaround is below)
• Dokkatoo/Dokka has problems with memory usage - still investigating

---

I found a bug in Dokkatoo that explains why some dependentSourceSets were missing from the dokka-configuration.json used by Dokkatoo

The getName() of all DokkaSourceSetIdSpecs is only returning the scope

dokkatoo/modules/dokkatoo-plugin/src/main/kotlin/dokka/parameters/DokkaSourceSetIdSpec.kt

Lines 37 to 38 in c74dd56

	@Internal
	override fun getName(): String = scopeId

So when two are put into the NamedDomainObjectContainer, only one will remain.

The other issue is that the Kotlin Native files are missing from the classpath. This can be fixed by adding a dependency onto the appropriate Configuration, kotlinCompilation.defaultSourceSet.implementationMetadataConfigurationName, however this triggers a bunch of the same "Reason: Task uses this output of task ':transformIosMainCInteropDependenciesMetadataForIde' without" errors that you're seeing in cashapp/redwood#823 (comment)

I'm inclined to believe this is caused by improper configuration in the Kotlin Gradle Plugin. The transformXYZCInteropDependenciesMetadataForIde task is dumping files into a directory, and some other task is using those same files.

This is not an easy problem to fix (or at least, not an easy problem to figure out how to fix - KGP is complicated!). The task is breaking project-isolation and not sharing files using Configurations.

So, as a workaround, the task can be manually disabled

tasks.matching {
  "CInteropMetadataDependencyTransformationTask" in (it::class.qualifiedName ?: "")
}.configureEach {
  enabled = false
}

Perhaps there's a clever way of disabling the task when the documentation is being generated?

tasks.matching {
  "CInteropMetadataDependencyTransformationTask" in (it::class.qualifiedName ?: "")
}.configureEach {
  // disable the IDE task when generating documentation
  enabled = gradle.taskGraph.allTasks.none {
      it is dev.adamko.dokkatoo.tasks.DokkatooTask || it is org.jetbrains.dokka.gradle.AbstractDokkaTask
    }
}

But the last hurdle... when I try disabling the CInteropDependenciesMetadataForIde tasks and run Dokkatoo, then Dokka requires HUGE amounts of memory and it crashes. I'm not sure what the problem is...

[aSemy]

This should be fixed by #76. Redwood managed to get Dokka working, so I'll close this issue.

If there are any more improvements or problems with how Dokkatoo handles Android projects, leave a comment or make a new issue.