Details

    • Similar Issues:

      Description

      The following configuration of the android-maven-plugin will not find the running emulator on Jenkins.
      Commenting this line will find the emulator.
      On my local machine this is working fine, but not in Jenkins.
      Both machines use maven 3.0.4 on Linux/Ubuntu.

      <groupId>com.jayway.maven.plugins.android.generation2</groupId>
      <artifactId>android-maven-plugin</artifactId>
      <configuration>
      ...
      <device>emulator</device>
      ...
      </configuration>
      </plugin>

      This configuration is useful on the local machine, if you are developing while charging your phone via USB.
      Best regards, that's a really nice plugin!
      Sebl29

        Attachments

          Activity

          sebl29 Sebastian Schefczyk created issue -
          Hide
          oldelvet Richard Mortimer added a comment -

          I suspect this may be due to the android-emulator-plugin starting a new ADB instance for each running job. This means that the android-maven-plugin may not find the ADB instance that contains the emulator.

          I have had a niggling suspicion that this may be the case for a while because I have seen matrix based testing fail at times and I suspect this happens when two emulators/ADB are running on the same slave at once.

          It may be that android-maven-plugin never finds a second ADB instance running on a non-standard port and hence if there is any other (non-Jenkins related) ADB hanging around things will fail.

          You may find that adding

          -Dandroid.device=${ANDROID_AVD_DEVICE}
          

          to the jobs maven goals and options configuration will fix it (this specifies the exact emulator device to use). However I suspect that may fail if my hypothesis about my matrix build failures is correct.

          Show
          oldelvet Richard Mortimer added a comment - I suspect this may be due to the android-emulator-plugin starting a new ADB instance for each running job. This means that the android-maven-plugin may not find the ADB instance that contains the emulator. I have had a niggling suspicion that this may be the case for a while because I have seen matrix based testing fail at times and I suspect this happens when two emulators/ADB are running on the same slave at once. It may be that android-maven-plugin never finds a second ADB instance running on a non-standard port and hence if there is any other (non-Jenkins related) ADB hanging around things will fail. You may find that adding -Dandroid.device=${ANDROID_AVD_DEVICE} to the jobs maven goals and options configuration will fix it (this specifies the exact emulator device to use). However I suspect that may fail if my hypothesis about my matrix build failures is correct.
          Hide
          oldelvet Richard Mortimer added a comment -

          Looking at recent ddmlib.jar (the library that android-maven-plugin uses to talk to ABD) source code it looks like the only way to change the ADB port number is by setting environment variable
          ANDROID_ADB_SERVER_PORT to the required port number before invoking maven. As per source code at com.android.ddmlib.AndroidDebugBridge#determineAndValidateAdbPort() from sdk/ddms/libs/ddmlib in the Android Git repos.

          Show
          oldelvet Richard Mortimer added a comment - Looking at recent ddmlib.jar (the library that android-maven-plugin uses to talk to ABD) source code it looks like the only way to change the ADB port number is by setting environment variable ANDROID_ADB_SERVER_PORT to the required port number before invoking maven. As per source code at com.android.ddmlib.AndroidDebugBridge#determineAndValidateAdbPort() from sdk/ddms/libs/ddmlib in the Android Git repos.
          Hide
          oldelvet Richard Mortimer added a comment -

          Please disregard my previous two comments. The plugin does seem to already ANDROID_ADB_SERVER_PORT so that should just magically work.

          Show
          oldelvet Richard Mortimer added a comment - Please disregard my previous two comments. The plugin does seem to already ANDROID_ADB_SERVER_PORT so that should just magically work.
          Hide
          oldelvet Richard Mortimer added a comment -

          So I decided to investigate this a bit more and took the radical step of setting -Dandroid.device=emulator in my Jenkins maven commandline. The good news is I get the same error here plus I've got a build environment setup for android-maven-plugin so I should be able to dig a little deeper over the next couple of days.

          For the record I get the following stack trace with plugin a-m-p 3.1.1 (get similar with 3.0.0 too).

          [INFO] Waiting for initial device list from the Android Debug Bridge
          [INFO] Found 1 devices connected with the Android Debug Bridge
          [INFO] android.device parameter set to emulator
          
          ...snip...
          
          [INFO] ------------------------------------------------------------------------
          [INFO] BUILD FAILURE
          [INFO] ------------------------------------------------------------------------
          [INFO] Total time: 1:01.522s
          [INFO] Finished at: Mon Feb 20 15:20:59 GMT 2012
          [INFO] Final Memory: 15M/495M
          [INFO] ------------------------------------------------------------------------
          mavenExecutionResult exceptions not empty
          message : Failed to execute goal com.jayway.maven.plugins.android.generation2:android-maven-plugin:3.1.1:internal-pre-integration-test (default-internal-pre-integration-test) on project mobilemax-it: No device found for android.device=emulator
          cause : No device found for android.device=emulator
          Stack trace : 
          org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal com.jayway.maven.plugins.android.generation2:android-maven-plugin:3.1.1:internal-pre-integration-test (default-internal-pre-integration-test) on project mobilemax-it: No device found for android.device=emulator
          	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:217)
          	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
          	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
          	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
          	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
          	at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
          	at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
          	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:319)
          	at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
          	at org.jvnet.hudson.maven3.launcher.Maven3Launcher.main(Maven3Launcher.java:79)
          	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
          	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
          	at java.lang.reflect.Method.invoke(Unknown Source)
          	at org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Launcher.java:329)
          	at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:239)
          	at org.jvnet.hudson.maven3.agent.Maven3Main.launch(Maven3Main.java:158)
          	at hudson.maven.Maven3Builder.call(Maven3Builder.java:104)
          	at hudson.maven.Maven3Builder.call(Maven3Builder.java:70)
          	at hudson.remoting.UserRequest.perform(UserRequest.java:118)
          	at hudson.remoting.UserRequest.perform(UserRequest.java:48)
          	at hudson.remoting.Request$2.run(Request.java:287)
          	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
          	at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
          	at java.util.concurrent.FutureTask.run(Unknown Source)
          	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
          	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
          	at java.lang.Thread.run(Unknown Source)
          Caused by: org.apache.maven.plugin.MojoExecutionException: No device found for android.device=emulator
          	at com.jayway.maven.plugins.android.AbstractAndroidMojo.doWithDevices(AbstractAndroidMojo.java:602)
          	at com.jayway.maven.plugins.android.AbstractAndroidMojo.undeployApk(AbstractAndroidMojo.java:644)
          	at com.jayway.maven.plugins.android.AbstractAndroidMojo.undeployApk(AbstractAndroidMojo.java:628)
          	at com.jayway.maven.plugins.android.AbstractAndroidMojo.deployDependencies(AbstractAndroidMojo.java:538)
          	at com.jayway.maven.plugins.android.phase11preintegrationtest.InternalPreIntegrationTestMojo.execute(InternalPreIntegrationTestMojo.java:36)
          	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101)
          	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:209)
          	... 27 more
          
          Show
          oldelvet Richard Mortimer added a comment - So I decided to investigate this a bit more and took the radical step of setting -Dandroid.device=emulator in my Jenkins maven commandline. The good news is I get the same error here plus I've got a build environment setup for android-maven-plugin so I should be able to dig a little deeper over the next couple of days. For the record I get the following stack trace with plugin a-m-p 3.1.1 (get similar with 3.0.0 too). [INFO] Waiting for initial device list from the Android Debug Bridge [INFO] Found 1 devices connected with the Android Debug Bridge [INFO] android.device parameter set to emulator ...snip... [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 1:01.522s [INFO] Finished at: Mon Feb 20 15:20:59 GMT 2012 [INFO] Final Memory: 15M/495M [INFO] ------------------------------------------------------------------------ mavenExecutionResult exceptions not empty message : Failed to execute goal com.jayway.maven.plugins.android.generation2:android-maven-plugin:3.1.1:internal-pre-integration-test ( default -internal-pre-integration-test) on project mobilemax-it: No device found for android.device=emulator cause : No device found for android.device=emulator Stack trace : org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal com.jayway.maven.plugins.android.generation2:android-maven-plugin:3.1.1:internal-pre-integration-test ( default -internal-pre-integration-test) on project mobilemax-it: No device found for android.device=emulator at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:217) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59) at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183) at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161) at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:319) at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156) at org.jvnet.hudson.maven3.launcher.Maven3Launcher.main(Maven3Launcher.java:79) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Launcher.java:329) at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:239) at org.jvnet.hudson.maven3.agent.Maven3Main.launch(Maven3Main.java:158) at hudson.maven.Maven3Builder.call(Maven3Builder.java:104) at hudson.maven.Maven3Builder.call(Maven3Builder.java:70) at hudson.remoting.UserRequest.perform(UserRequest.java:118) at hudson.remoting.UserRequest.perform(UserRequest.java:48) at hudson.remoting.Request$2.run(Request.java:287) at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang. Thread .run(Unknown Source) Caused by: org.apache.maven.plugin.MojoExecutionException: No device found for android.device=emulator at com.jayway.maven.plugins.android.AbstractAndroidMojo.doWithDevices(AbstractAndroidMojo.java:602) at com.jayway.maven.plugins.android.AbstractAndroidMojo.undeployApk(AbstractAndroidMojo.java:644) at com.jayway.maven.plugins.android.AbstractAndroidMojo.undeployApk(AbstractAndroidMojo.java:628) at com.jayway.maven.plugins.android.AbstractAndroidMojo.deployDependencies(AbstractAndroidMojo.java:538) at com.jayway.maven.plugins.android.phase11preintegrationtest.InternalPreIntegrationTestMojo.execute(InternalPreIntegrationTestMojo.java:36) at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:209) ... 27 more
          oldelvet Richard Mortimer made changes -
          Field Original Value New Value
          Assignee Christopher Orr [ orrc ] Richard Mortimer [ oldelvet ]
          Hide
          oldelvet Richard Mortimer added a comment -

          After adding some instrumentation to android-maven-plugin it seems that ddmlib.jar (the Android library that interfaces with the emulator/device) is not reporting the emulator as being an emulator.

          [INFO] Waiting for initial device list from the Android Debug Bridge
          [INFO] Found 1 devices connected with the Android Debug Bridge
          [INFO] android.device parameter set to emulator
          [INFO] checking device=localhost_47048_unknown_sdk, port=58222, name=null, isEmulator=false
          

          Looking at the ddmlib.jar source code the isEmulator method detects an emulator based on the serial number of the device. In particular it expects it to confirm to the following regexp pattern.

              /** Emulator Serial Number regexp. */
              final static String RE_EMULATOR_SN = "emulator-(\\d+)"; //$NON-NLS-1$
          
          

          android-emulator-plugin seems to use a different serial number format to manually launched emulator instances. Manual launched serial numbers resemble emulator-5554 whereas the android emulator version is localhost:5554.

          So far I have not identified where the serial number is generated/set but it does look like this issue comes down to the serial number format.

          Show
          oldelvet Richard Mortimer added a comment - After adding some instrumentation to android-maven-plugin it seems that ddmlib.jar (the Android library that interfaces with the emulator/device) is not reporting the emulator as being an emulator. [INFO] Waiting for initial device list from the Android Debug Bridge [INFO] Found 1 devices connected with the Android Debug Bridge [INFO] android.device parameter set to emulator [INFO] checking device=localhost_47048_unknown_sdk, port=58222, name= null , isEmulator= false Looking at the ddmlib.jar source code the isEmulator method detects an emulator based on the serial number of the device. In particular it expects it to confirm to the following regexp pattern. /** Emulator Serial Number regexp. */ final static String RE_EMULATOR_SN = "emulator-(\\d+)" ; //$NON-NLS-1$ android-emulator-plugin seems to use a different serial number format to manually launched emulator instances. Manual launched serial numbers resemble emulator-5554 whereas the android emulator version is localhost:5554 . So far I have not identified where the serial number is generated/set but it does look like this issue comes down to the serial number format.
          Hide
          oldelvet Richard Mortimer added a comment -

          Problem found. Unfortunately it is deep in the emulator (qemu) startup code. The following code registers the emulator instance with the adb server.

          qemu-setup.c:
            sprintf(tmp,"0012host:emulator:%d",base_port+1);
            socket_send(s, tmp, 18+4);
          

          This registers the emulator with the adb server but it makes two assumptions:

          1. that the user port is one less than the adb port for the emulator. Comments in the code also suggest that the user port was once assumed to be an even number. The server takes the adb port number supplied and subtracts one to get the user port. It then tries to connect to the user port to confirm that it really is an emulator.
          2. only 4 characters of the adb port number is sent to the adb server (the +4 in socket_send). Thus if the port number is greater than 9999 or less than 1000 then the wrong port number is sent.

          I haven't searched the Android bug database for this yet but will search/report there later today.

          However that does not make things work for a-e-p because we will need to support older emulators.

          I tested a fix that uses the Jenkins port allocator to allocate 7000 for user port, 7001 for emulator adb port and then 8000 for the adb server. With that maven happily worked using -Dandroid.device=emulator.

          [INFO] Found 1 devices connected with the Android Debug Bridge
          [INFO] android.device parameter set to emulator
          [INFO] checking device=emulator-6000_hudson_en-GB_160_WVGA_android-7_unknown_sdk, port=8000, name=hudson_en-GB_160_WVGA_android-7, isEmulator=true
          [INFO] Emulator emulator-6000_hudson_en-GB_160_WVGA_android-7_unknown_sdk found.
          

          Obviously to provide a proper fix we need to do some magic with the port allocator plugin. Assuming that it is acceptable that we require a newer port allocator plugin for a-e-p then I'll look to extend that to help us. Failing that we can likely make the existing allocator work with a bit of messing around.

          Show
          oldelvet Richard Mortimer added a comment - Problem found. Unfortunately it is deep in the emulator (qemu) startup code. The following code registers the emulator instance with the adb server. qemu-setup.c: sprintf(tmp, "0012host:emulator:%d" ,base_port+1); socket_send(s, tmp, 18+4); This registers the emulator with the adb server but it makes two assumptions: that the user port is one less than the adb port for the emulator. Comments in the code also suggest that the user port was once assumed to be an even number. The server takes the adb port number supplied and subtracts one to get the user port. It then tries to connect to the user port to confirm that it really is an emulator. only 4 characters of the adb port number is sent to the adb server (the +4 in socket_send). Thus if the port number is greater than 9999 or less than 1000 then the wrong port number is sent. I haven't searched the Android bug database for this yet but will search/report there later today. However that does not make things work for a-e-p because we will need to support older emulators. I tested a fix that uses the Jenkins port allocator to allocate 7000 for user port, 7001 for emulator adb port and then 8000 for the adb server. With that maven happily worked using -Dandroid.device=emulator . [INFO] Found 1 devices connected with the Android Debug Bridge [INFO] android.device parameter set to emulator [INFO] checking device=emulator-6000_hudson_en-GB_160_WVGA_android-7_unknown_sdk, port=8000, name=hudson_en-GB_160_WVGA_android-7, isEmulator= true [INFO] Emulator emulator-6000_hudson_en-GB_160_WVGA_android-7_unknown_sdk found. Obviously to provide a proper fix we need to do some magic with the port allocator plugin. Assuming that it is acceptable that we require a newer port allocator plugin for a-e-p then I'll look to extend that to help us. Failing that we can likely make the existing allocator work with a bit of messing around.
          Hide
          oldelvet Richard Mortimer added a comment -

          Proof of concept fix committed to my personal github a-e-p fork.
          https://github.com/oldelvet/android-emulator-plugin/tree/jenkins-12821-poc

          I intend to implement the port allocator side of this in due course but have verified that the fix allows android.device=emulator to work with the proof of concept.

          Show
          oldelvet Richard Mortimer added a comment - Proof of concept fix committed to my personal github a-e-p fork. https://github.com/oldelvet/android-emulator-plugin/tree/jenkins-12821-poc I intend to implement the port allocator side of this in due course but have verified that the fix allows android.device=emulator to work with the proof of concept.
          Hide
          orrc Christopher Orr added a comment -

          Great stuff.

          Did you file that Android bug, out of interest?

          Show
          orrc Christopher Orr added a comment - Great stuff. Did you file that Android bug, out of interest?
          Hide
          oldelvet Richard Mortimer added a comment -

          No I didn't get around to filing it. With the new SDK being out I figure I ought to test against that before reporting it now.

          Show
          oldelvet Richard Mortimer added a comment - No I didn't get around to filing it. With the new SDK being out I figure I ought to test against that before reporting it now.
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Richard Mortimer
          Path:
          src/main/java/org/jvnet/hudson/plugins/port_allocator/PortAllocationManager.java
          src/test/java/org/jvnet/hudson/plugins/port_allocator/PortAllocationManagerTest.java
          http://jenkins-ci.org/commit/port-allocator-plugin/007ca2616293ece2e033070e4e0ad13253eecd0a
          Log:
          Add PortAllocationManager#allocateConsecutivePortRange to allow multiple
          ports to be allocated together within a specific range of ports.
          This forms part of the fix for JENKINS-12821 in android-emulator-plugin..

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Richard Mortimer Path: src/main/java/org/jvnet/hudson/plugins/port_allocator/PortAllocationManager.java src/test/java/org/jvnet/hudson/plugins/port_allocator/PortAllocationManagerTest.java http://jenkins-ci.org/commit/port-allocator-plugin/007ca2616293ece2e033070e4e0ad13253eecd0a Log: Add PortAllocationManager#allocateConsecutivePortRange to allow multiple ports to be allocated together within a specific range of ports. This forms part of the fix for JENKINS-12821 in android-emulator-plugin..
          Hide
          g_gabber Edgar H added a comment - - edited

          When this bug should be fixed?
          Maybe it is possible to set a custom ports?
          It is pity, such a great plugin, and I cant use it with maven

          Show
          g_gabber Edgar H added a comment - - edited When this bug should be fixed? Maybe it is possible to set a custom ports? It is pity, such a great plugin, and I cant use it with maven
          Hide
          oldelvet Richard Mortimer added a comment -

          @Christopher Orr I finally managed to find time to log the issue in the AOSP issue tracker.

          http://code.google.com/p/android/issues/detail?id=60024

          Show
          oldelvet Richard Mortimer added a comment - @Christopher Orr I finally managed to find time to log the issue in the AOSP issue tracker. http://code.google.com/p/android/issues/detail?id=60024
          Hide
          oldelvet Richard Mortimer added a comment -

          For reference the fix to workaround the broken AOSP functionality is at
          https://github.com/jenkinsci/android-emulator-plugin/pull/27

          Show
          oldelvet Richard Mortimer added a comment - For reference the fix to workaround the broken AOSP functionality is at https://github.com/jenkinsci/android-emulator-plugin/pull/27
          Hide
          orrc Christopher Orr added a comment -

          Version 2.11 of the plugin has been released including these fixes

          Show
          orrc Christopher Orr added a comment - Version 2.11 of the plugin has been released including these fixes
          orrc Christopher Orr made changes -
          Status Open [ 1 ] Closed [ 6 ]
          Resolution Fixed [ 1 ]
          Hide
          randroid Roberto Andrade added a comment -

          I just upgraded to 2.11(.1) specifically because I started having this issue recently and am now facing another weird one which seems related:

          After the plugin starts the emulator, its subsequent attempts to "connect" to it fail and ends up timing out. I see the emulator window available, but the sequence of "adb connect emulator-xxxx" and "adb -s emulator-xxxx shell getprop dev.bootcomplete" keeps on failing with a "error: device offline" error.

          any tips as to what could be the problem?

          Show
          randroid Roberto Andrade added a comment - I just upgraded to 2.11(.1) specifically because I started having this issue recently and am now facing another weird one which seems related: After the plugin starts the emulator, its subsequent attempts to "connect" to it fail and ends up timing out. I see the emulator window available, but the sequence of "adb connect emulator-xxxx" and "adb -s emulator-xxxx shell getprop dev.bootcomplete" keeps on failing with a "error: device offline" error. any tips as to what could be the problem?
          Hide
          randroid Roberto Andrade added a comment -

          I reopened https://issues.jenkins-ci.org/browse/JENKINS-11952 as it seems to be a better fit for the issue description, but I think it might have been caused by the changes that fixed this issue.

          Show
          randroid Roberto Andrade added a comment - I reopened https://issues.jenkins-ci.org/browse/JENKINS-11952 as it seems to be a better fit for the issue description, but I think it might have been caused by the changes that fixed this issue.
          rtyler R. Tyler Croy made changes -
          Workflow JNJira [ 143279 ] JNJira + In-Review [ 205689 ]

            People

            • Assignee:
              oldelvet Richard Mortimer
              Reporter:
              sebl29 Sebastian Schefczyk
            • Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: